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Directions 


In this booklet, you will find questions about yourself. Some questions ask for facts 
while other questions ask for your opinion. 


Each question is followed by a number of answers. Fill in the oval next to or under 
the answer of your choice as shown in the example below. 


Example 
How often do you do these things? 


Fill in only one oval for each row. 


Every day Once or Once or Never or 
or almost twicea twicea almost 
every day week month never 

a) I talk with my friends ------------------ @ ‘e>) @ 

b) I play sports ------------------------------- Oo) ) @ 

c) I listen to music -------------------------- oo) @ ®@ 


e Read each question carefully, and pick the answer you think is best. 
e Fill in the oval next to or under your answer. 


e Ifyou decide to change your answer, completely erase your first choice. 


Then, fill in the oval next to or under your new answer. 


e Ask for help if you do not understand something or are not sure how to answer. 
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About You 


1 


A. Are you female or male? 


Fill in one oval only. 
Female -- © 


Male -- @ 


B. Are you Hispanic or Latino? 


Fill in one oval only. 
Yes, Iam Hispanic or Latino -- © 


No, I am not Hispanic or Latino -- @ 


C. Which of the following best describes you? 


Fill in ovals for all that apply. 


White -- © 

Black or African American -- © 

Asian -- © 

American Indian or Alaska Native -- © 
Native Hawaiian or other 

Pacific Islander -- © 
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2 


When were you born? 


Fill in the ovals next to the month and year you were born. 


a) Month 


January --- 
February --- 
March --- 
April --- 
May --- 

June --- 

July --- 
August --- 
September --- 
October --- 
November --- 


December --- 


®@ 6 


0680066006060 8 0 


b) Year 


1998 --- 
1994 --- 
1995 --- 
1996 --- 
1997 --- 
1998 --- 
1999 --- 
2000 --- 
2001 --- 
Other --- 


80 © 8 6 © 8 0 


Q @ 
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3 


A. How often do you speak English at home? 
Fill in one oval only. 
Always -- © If Always, please go to question 4 
Almost always -- @ 
Sometimes -- © 


Never --@ 


If Almost always, Sometimes, Never, 
please go to question 3B 


B. What language do you speak at home (other than 
English)? 
Fill in one oval only. 
Spanish -- @ 


Other --@_ Please specify 
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A 


How many days were you absent from school in the 
last month? 


Fill in one oval only. 


None --@ 

1 or 2 days--@ 

3 or 4 days --® 

5 to 10 days --@ 

More than 10 days --® 


5) 


Have you ever repeated a grade? 


Fill in only one oval for each row. 


Yes No 
a) In elementary school -------------------- (@) @ 
b) In middle or junior high school ------ fo) @ 
c) In high school ----------------------------- @® @ 
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6 


About how many books are there in your home? (Do not 
count magazines, newspapers, or your school books.) 


Fill in one oval only. 


None or very few 
(0-10 books) -- @ 


Enough to fill one shelf 
(11-25 books) -- @ 


Enough to fill one bookcase 
(26-100 books) -- @ 


Enough to fill two bookcases 
(101—200 books) -- @ 


Enough to fill three or more bookcases 
(more than 200) --@ 


How many digital information devices are there in your 
home? Count computers, tablets, smartphones, smart TVs, 
and e-readers. (Do not count other devices.) 


Fill in one oval only. 


None -- @ 

1-3 devices --@ 

4-6 devices --@ 

7-10 devices -- @ 

More than 10 devices -- © 
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8 


Do you have any of these things? 


Fill in only one oval for each row. 


Yes No 

a) Your own computer ----------------------- @O fe 

a) Yow awa ta blet<2225422 4h sect cecececccse @o fo 

c) Your own smartphone -------------------- @O foo) 

d) Your own graphing calculator---------- @ re) 
e) A gaming system 

(e.g., PlayStation, Wii, Xbox) ----------- a @ 

f) Study desk/table for your use ---------- @ @ 

@) Your Own POOM--e+sece-edenseeueraseeeecnsne ‘) ren) 

Ih) “Vow Gwit car a-sesecceeseceetaert arse @O Om 
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9 


A. What is the highest level of education completed by 
your mother (or stepmother or female legal guardian)? 


Fill in one oval only. 
Less than high school -- © 
Some high school -- 
High school graduate -- 
Associate’s degree (2-year college program) -- 
Bachelor’s degree (4-year college program) -- 


Master’s degree or professional 
degree (MD, DDS, lawyer, minister) -- © 


Doctorate (Ph.D., or Ed.D.) -- @ 


I don’t know -- 


B. What is the highest level of education completed by 
your father (or stepfather or male legal guardian)? 


Fill in one oval only. 


Less than high school -- © 

Some high school -- 

High school graduate -- 

Associate’s degree (2-year college program) -- 
Bachelor’s degree (4-year college program) -- 


Master’s degree or professional 
degree (MD, DDS, lawyer, minister) -- © 


Doctorate (Ph.D., or Ed.D.) -- @ 


I don’t know -- 
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10 


What kind of work do your father (or stepfather or male 

legal guardian) and mother (or stepmother or female legal 
guardian) do for their main jobs? 

For each, fill in the oval for the job category that best describes what he/she 
does. Each category has a few examples to help you decide the correct category. 


If your father or mother is not working now, think about the last job he/she had. 


Fill in only one oval for each column. 


Your Your 
father mother 
a) Has never worked for pay ----------- @ @ 
b) Small Business Owner --------------- o) @ 


Includes owners of small businesses 
(fewer than 25 employees) such as 
retail shops, services, restaurants 


c) Clerk --------------------+--2-2--2--20-2-0- o>) @® 
Includes office clerks; secretaries; typists; 
data entry operators; customer service 
clerks 


d) Service or Sales Worker -------------- ) @ 


Includes travel attendants; restaurant 
service workers; personal care workers; 
protective service workers; enlisted 
military and police; salespersons; street 
vendors 


e) Skilled Agricultural or 
Fishery Worker ------------------------- ro) ) 


Includes farmers; forestry workers; 
fishery workers; hunters and trappers 


f) Craft or Trade Worker ---------------- ep) o>) 


Includes builders, carpenters, plumbers, 
electricians, metal workers; machine 
mechanics; handicraft workers 


Continued on next page ==» 
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1 O (continued) 


g) Plant or Machine Operator ------------- 


Includes plant and machine operators; 
assembly-line operators; motor-vehicle 
drivers 


h) General Laborers --------------------------- 


Includes domestic helpers and cleaners; 
building caretakers; messengers, porters, 
and doorkeepers; farm, fishery, 
agricultural, and construction workers 


1) Corporate Manager or 
Senior Official -----------------------------+- 


Includes corporate managers such as 
managers of large companies (25 or more 
employees) or managers of departments 
within large companies; legislators or 
senior government officials; senior officials 
of special-interest organizations; military 
officers 


j) Professional ---------------------------------- 


Includes scientists; mathematicians; 
computer scientists; architects; engineers; 
life science and health professionals; 
teachers; legal professionals; social 
scientists; writers and artists; religious 
professionals 


k) Technician or Associate 
Professional ---------------------+-+--+--+---- 


Includes science, engineering, and 
computer associates and technicians; 

life science and health technicians and 
assistants; teacher aides; finance and 
sales associate professionals; business 
service agents; administrative assistants 


1) I don’t know ---------------------------------- 


Your Your 
father mother 

() (>) 

@O @ 

@O © 

(>) @ 

(>) (o>) 

o> (o>) 
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11 


How far in your education do you expect to go? 


High school -- 


Associate’s degree 


(2-year college program) -- 


Bachelor’s degree 


(4-year college program) -- 


Master’s degree or professional degree 


(MD, DDS, lawyer, minister) -- 
Doctorate (Ph.D., or Ed.D.) -- 
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12 


If you plan to continue your education, which area(s) do 
you intend to study? 


Fill in ovals for all that apply. 


a) Mathematics or Statistics ------------------------------- >) 
b) Physics ----------------------2---2-2-2- 2c rere ec ececec reece eee ‘o) 
c) Chemistry ----------------------2-22- 2 rc ene cere eee ce reece ences ) 


d) Biological and Biomedical Sciences (e.g., 
dentistry, medicine, nursing, pharmacology, 
veterinary medicine) -------------------------------------- @ 


e) Engineering and Engineering Technologies 
(e.g., aerospace engineering, chemical 
engineering, civil engineering, electrical 


engineering, mechanical engineering) -------------- ‘o) 
f) Computer and Information Sciences ---------------- @ 
g) Education --------------------------- 22-22-0222 nen n ee ee nee nee @ 
h) Business (e.g., accounting, marketing, 
administration, finance, management) ------------- ) 
1) Law------------------------220 renee ncn n ence rence ec ne ne ece cee O) 
j) Social Sciences (e.g., sociology, political 
science, economics, psychology) ----------------------- oO) 
k) Arts and Humanities (e.g., art, language, 
literature, history, philosophy) ------------------------ @ 
1) Other Science Fields of Study ------------------------- @ 
m) Other Non-science Fields of Study ------------------- @ 
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13 


In the future, do you want to work in any of the 


following professional fields? 


a) 
b) 


c) 


d) 
e) 


h) 


1) 


Fill in only one oval for each row. 


Education (e.g., teacher, university professor) 


Engineering and Engineering Technologies 
(e.g., aerospace engineer, chemical 
engineer, civil engineer, electrical 


engineer, mechanical engineer) ------------------------ 


Computer and Information Sciences 
(e.g., database administrator, network 
administrator, software or application 


developer, systems analyst) ----------------------------- 


Finance/Banking -------------------------------2---2-2-20--- 


Biological and Biomedical Sciences 
(e.g., biomedical engineer, biochemist, 
biophysicist, dentist, medical doctor, nurse, 


veterinarian) -------------------2-2---2-2- 2 rere rere ec ee ee ree 
Environmental Sciences ------------------+------2-+----- 


Agriculture and Agricultural Sciences --------------- 


Actuarial Sciences (i.e., uses mathematical 


and statistical methods to assess risk) --------------- 


Other Fields --------------------------2-02-2-2e0re nc eee reece 


Maybe 


o>) 


0 
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14 


A. Was your mother (or stepmother or female legal 
guardian) born in the United States? (“United States” 
includes the 50 states, its territories, the District of 
Columbia, and U.S. military bases abroad.) 


Fill in one oval only. 


No-- @ 


I don’t know -- © 


B. Was your father (or stepfather or male legal guardian) 
born in the United States? 


Fill in one oval only. 


No-- @ 


I don’t know -- © 
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15 


A. Were you born in the United States? 
Fill in one oval only. 


Yes -- © 
(If Yes, go to question 16) 


If No, 


B. If you were not born in the United States, how old 
were you when you came to the United States? 


Fill in one oval only. 
Older than 15 years old -- © 
11 to 15 years old -- 
5 to 10 years old -- 


Younger than 5 years old -- 
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16 


How much time do you spend in physics class each week? 


minutes per week 
Write in the number of minutes per week. 


Please convert the number of classes/periods into minutes. 


17 


How much time do you spend on physics outside of class 
each week? 


minutes per week 
Write in the number of minutes per week. 


Please convert the number of hours into minutes. 


18 


A. During the school year, do you work at a paid job on 
a regular basis? 


Fill in one oval only. 


(If No, go to question 19) 


If Yes, 


B. How much time do you spend working at the paid job 
each week? 


minutes per week 
Write in the number of minutes per week. 


Please convert the number of hours into minutes. 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide 


D-340 


Exhibit D-14. TIMSS Advanced 2015 Physics Student Questionnaire—Continued 


19 


A. During the last 12 months, have you attended 
extra lessons or tutoring not provided by the school in 


physics? 
Fill in one oval only. 
Yes -- © 
No-- @ 
(If No, go to question 20) 
If Yes, 


B. Why did you attend these extra lessons or tutoring? 


Fill in only one oval for each row. 


Yes No 
a) To excel in class -------------------------- o @ 
b) To keep up in class ---------------------- @ @ 
c) To do well on an examination -------- o) @ 


C. For how many of the last 12 months have you 
attended extra lessons or tutoring in physics? 


Fill in one oval only. 
Less than 4 months -- © 
4-8 months -- © 


More than 8 months -- ® 
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20 


How much do you agree with these statements about 


your physics lessons? 


a) 


b) 


g) 


h) 
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Fill in only one oval for each row. 


Agree 
a lot 


The teacher clearly communicates 

the purpose of each physics 

leSSON. ------+--+----2--2-02-0 ee eee eee eee ee eee ‘o) 
I know what my teacher 

expects me to do ------------------------- ) 


My teacher is easy to understand -- © 


I am interested in what my 
teacher says ------------------------------- oo) 


My teacher gives me interesting 
things to do -------------------------------- o) 


My teacher asks me thought- 
provoking questions -------------------- ‘o) 


My teacher has clear answers to 
my questions ------------------------------ ) 


My teacher links new content to 
what I already know ------------------- ‘) 


Disagree 
a lot 
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2 O (continued) 


How much do you agree with these statements about 
your physics lessons? 


Fill in only one oval for each row. 


Agree Agree Disagree Disagree 
a lot a little a little a lot 
i) My teacher is good at explaining 
PHYSICS -----+----+--00e-cneneneeneneneenone @D o>) ®D ) 
j) My teacher provides the 
opportunity for me to show what 
I have learned --------------------------- o) @ ® ® 


k) My teacher encourages me to keep 

working on physics 

problems until I solve them --------- o) @ ® ® 
1) My teacher provides helpful 


feedback on my schoolwork 
(including homework) ----------------- ) @ ® ® 


m) My teacher uses a variety of 
teaching methods, tasks, and 
activities to help us learn ------------ o) @ o>) ® 


n) My teacher believes that I can 
learn difficult physics 
material ----------------------------------- o) @ ® cp) 


o) I like the way my teacher 
teaches physics ------------------------- o) o) o>) @ 
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21 


Do you use the Internet to do any of the following tasks 
for physics schoolwork (including classroom tasks, 


homework, and studying outside of class)? 


a) 


b) 


e) 


g) 
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Access the textbook or other 


course materials ------------------------- 


Access assignments posted online 


by my teacher ---------------------------- 


Collaborate with classmates on 
physics assignments or 


Projects ----------------------2-----2-20----- 


Communicate with the teacher ----- 


Discuss physics topics with 


other students --------------------------- 


Find information, articles, or 
tutorials to aid in understanding 


physics concepts ------------------------- 


Find information, articles, or 
tutorials to aid in solving 


physics problems. ----------------------- 


Fill in only one oval for each row. 


No 
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22 


How much do you agree with these statements about the 
physics you are studying? 


Fill in only one oval for each row. 


Agree Agree Disagree Disagree 
a lot a little a little a lot 

a) I enjoy conducting experiments 

or investigations in physics ---------- ‘o) @ ® ® 
b) I get a sense of satisfaction when 

I solve physics problems -------------- o) @ ® @® 
c) I feel bored when I do my 

physics schoolwork --------------------- oo) ® ® ® 
d) I like studying for my physics 

class outside of school ----------------- co) o) >) ® 
e) Itis interesting to learn 

physics laws and principles ---------- ) @ ep) ® 
f) I dread my physics class--------------- @ @ ® ® 
g) Iam studying physics because 

I like to learn new things-------------- o) @ o>) ® 
h) Ienjoy figuring out challenging 

physics ----------------+---+-----2---2--+--- @ @ ® ® 
1) Physics is one of my 

favorite subjects ------------------------- oO) ‘o) ® @ 
j) Jobs that require physics skills 

seem interesting to me ---------------- o) @ ® ® 
k) I wish I did not have to 

study physics ----------------------------- o) @ ® ® 


1) LT enjoy thinking about the world in 
terms of laws of physics --------------- ‘o) @ e) ® 
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23 


How much do you agree with these statements about the 
physics you are studying? 


Fill in only one oval for each row. 


Agree Agree Disagree Disagree 
a lot a little a little a lot 
a) Learning physics will help 
me get ahead in the world ------------ @ @ ® @® 
b) Itis important to do well in my 
physics class ------------------------------ Oo) @ ® ® 
c) The physics I am studying 
is not useful for my future ------------ @ @Q (e) @ 
d) My parents are pleased that Iam 
taking physics ---------------------------- o) @ ® @® 
e) Doing well in physics will 
help me get into the college or 
university of my choice ---------------- DO @ ® ® 


f) Learning physics does not seem to 
be a worthwhile exercise -------------- o) @ (©) ep) 


g) My parents think that it is 
important that I do well in my 
physics class ------------------------------ o) @ ® ® 


h) I like telling people I am studying 
PRY SICS tes sevee star ee nec eemnee ei ancertnenee 2) @ ® ® 


i) Learning physics will give me 
more job opportunities ----------------- o) @ e) ® 
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24 


How hard was this test compared to most other tests you 
have taken this year in school? 


Fill in one oval only. 
Easier than other tests -- © 
About as hard as other tests -- 


Harder than other tests -- 


eo 8 0 


Much harder than other tests -- 


25 


How hard did you try on this test compared to how hard 
you tried on most other tests you have taken this year in 
school? 


Fill in one oval only. 
Not as hard as on other tests-- © 
About as hard as on other tests-- 


Harder than on other tests-- 


© 0 0 


Much harder than on other tests-- 


26 


How important was it to you to do well on this test? 
Fill in one oval only. 


Not very important-- © 
Somewhat important-- 


Important-- 


© 0 0 


Very important-- 
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Academic and Post-Secondary 
Preparation 


27 


In what grade did you complete any of the courses listed below? 


Fillin one or more ovals in each row. 


Grade 8 Grade Grade Grade Grade 
Never or earlier 9 10 11 12 

a) General or unified science --@ oO) o) o) ©) ® 
b) Earth and space science --@ o) o) o) o) Oo 
c) Life science (other than 

biology) ------------------------- o) ® Oo) @®O Oo} <p) 
d) Physical science (other 

than chemistry or physics) --@® @ @ @ >) @ 
e) First-year biology ----------- o) ‘o) (@) @) @) o) 
f) Second-year biology -------- o) ) o) o) o) ) 
g) First-year chemistry ------- @ ‘o) @) (@) ) o) 
h) Second-year chemistry ----@ ‘) o) o) ) @ 
i) First-year physics ---------- @ o) o) o) o) o) 
j) Second-year physics ------- ) ‘o) (@) (@) ©) o) 
k) Engineering and 

technology --------------------- o) o) o> @O @®O <D) 
1) Other advanced 

science course ---------------- ) @ op) (o) o) oo) 
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28 


Please indicate if you have taken or are currently 
enrolled in any of the following Advanced Placement 
(AP) courses. Have taken or are enrolled in: 


Fill in only one oval for each row. 


Yes No 

a) Advanced Placement (AP) 
Biology ------------------------------------- op) @ 

b) Advanced Placement (AP) 
Environmental Science---------------- @ @ 

c) Advanced Placement (AP) 
Chemistry --------------------------------- ) o>) 

d) Advanced Placement (AP) 
Physics B or C---------------------------- o) @ 

e) Advanced Placement (AP) 
Computer Science A or AB----------- @ @ 


29 


Are you currently enrolled in or have you taken any 
online science courses? 


Fill in one oval only. 
No --@ 
Yes, but not for credit --@ 
Yes, for high school credit -- ® 
Yes, for college credit --@® 


Yes, for both high school 
and college credit -- 


© 
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30 


Are you currently enrolled in or have you taken an 
International Baccalaureate (IB) physics course? 


Fill in one oval only. 
Yes --@ 


No--@ 


31 


During this school year, which of the following have you 
done? 


Taken the SAT or ACT college 
entrance exams -- 


0 


Submitted the Free Application for 
Federal Student Aid (FAFSA) -- 


Applied to a 2-year college -- 
Been accepted to a 2-year college -- 


Applied to a 4-year college -- 


086 0 8 


Been accepted to a 4-year college -- 


Talked with a military recruiter or 
contacted a ROTC program -- 


0 


Enlisted in the military or enrolled 
in a ROTC program -- 


Applied for a full-time job -- 


Been interviewed for a full-time job -- 


8 0 0 0 


None of the above -- 
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Your School 


32 


What do you think about your school? Tell how much 


you agree with these statements. 


h) 


1) 
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Fill in only one oval for each row. 


Agree 
a lot 
I enjoy school ---------------------------- @ 
I feel safe when I am at school ----- oO) 


I feel like I belong at this school --- © 


I like to see my classmates 
at school --------------+--+--+----+--------+ <) 


Teachers at my school are 
fair to me --------------------------------- @ 


I am proud to go to this school ----- ‘o) 
1 earn a loti. school ‘x-<0cee<n-220e02s ‘-) 


My classmates respect students 
who excel in school subjects --------- o) 


My classmates respect students who 
struggle learning school subjects -- © 


Agree 
a little 


o>) 


o>) 


Disagree 
a little 


® 


® 


Disagree 


a lot 


® 


<>) 
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During this school year, how often have other students 
from your school done any of the following things to you 


(including through texting or the Internet)? 


g) 


h) 
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Made fun of me or 


called me names ------------------------ 
Excluded me from their activities -- 
Spread lies about me ------------------ 


Stole something from me ------------- 


Hit or hurt me (e.g., shoving, 


hitting, kicking) -----------------+---0+-- 


Made me do things I didn’t 


want to do --------------------------------- 


Posted embarrassing things 


about me online ------------------------- 


Threatened me -------------------------- 


At least 
once a 
week 


Once or 
twice 
a month 


8 8 © O 


Fill in only one oval for each row. 


A few 
times 
a year 


8 6 0 @ 


Never 
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34 


During this school year, did you participate in any of 
these extracurricular activities? 


Fill in ovals for all that apply. 
Sports -- © 
Performing arts -- 
Academic clubs -- 
Vocational/professional clubs -- 
Honor societies -- 
Publications -- 
Student government -- 


Service clubs -- 


OO 0:0 OO: 2 © 


Hobby clubs -- 


Thank you for filling out the questionnaire! 
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\ 


© IEA, 2015 


TIMSS Advanced 2015 
Curriculum Questionnaire— 
Mathematics 


Advanced 


Mathematics 


im TIMSS & PIRLS 
International Study Center 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSSA2015MS_OCQ - English 
You are not logged in. 


IEA, 


TIMSS 


A DATA PROCESSING 


AD RESEARCH CENTER 


2015 
Welcome to the IEA - DPC SurveySystem 


TIMSS Advanced 2015 
Curriculum Questionnaire 


Please enter your user ID and password (Checksum). 


User ID: 
Password: 


Login 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics 


The TIMSS Advanced 2015 Curriculum Questionnaires are designed to collect basic information about the structure of the education 
system as well as the organization, content, and implemeniation of the advanced matrematics and physics curricula in each country. 
There are separate questionnaires for Advanced Mathematics and Physics. 


The questionnaires should be completed by the National Research Coordinators, drawing on the expertise of curriculum specialists 
and educators. Please submit the questionnaires no later than August 31, 2015. 


To begin this questionnaire, please click on the “Next” button. When navigating through the questionnaire, make sure to confirm your 
responses by clicking on the “Next” or “Previous” button. Tc go to a particular section cr item, please click on the corresponding link in 
the “Table of Contents”. 


If you have any questions about the content of this questionnaire, please contact the TIMSS & PIRLS Internatione! Study Center at 
Boston College: timss@bc.edu 


If you have any technical questions on how to complete this questionnaire, please comact the IEA Data Processirg & Research Center 
(DPC): timss@iea-dpc.de 


Table of Contents Next 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - About the Advanced Mathematics Programs (Tracks) 


eB About the Advanced Mathematics Programs (Tracks) 


This questionnaire refers to the national advanced mathematics curriculum that was in effect for the students assessed in TIMSS 
Advanced 2015—the curriculum that covers advanced mathematics instruction for the majority of students in these programs or tracks. 
If you do not have a national curriculum, please summarize for your state or provincial curricula. 


1. A. Describe the advanced mathematics programs/tracks assessed by TIMSS Advanced 2015. How do 
the programs/tracks fit into the overall curriculum from the first grade through the final year? How do 
they relate with programs at the university level, if at all (e.g., is participation a prerequisite for studying 
certain fields such as engineering or medicine)? 


Examples of information reported for TIMSS Advanced 2008 can be found in the second column of Exhibit 1.1 on 'S 26-27 of the 
2008 report. Click here to view 


B. How many years are students in these programs/tracks, and at which grade do they start? 


Examples of information reported for TIMSS Advanced 2008 can be found in the third column of Exhibit 1.1 on pages 26-27 of the 
2008 report. Click here to view 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


C. What is the total amount of class time in advanced mathematics for the students in the advanced 
mathematics programs/tracks? 


Examples of information reported for TIMSS Advanced 2008 can be found in the fourth column of Exhibit 1.1 on pages 26-27 of the 
2008 report. Click here to view 


hours per year (1 hour = 60 minutes) 


Comments: 


| Previous 1/13 Table of Contents Next 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide D-358 


Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - Criteria for Admission 


_ Criteria for Admission 


2. A. What are the criteria for admission to these advanced mathematics programs/tracks? 


Examples of information reported for TIMSS Advanced 2008 can be found in the fifth column of Exhibit 1.1 on Ss 26-27 of the 2008 
report. Click here to view 


p 


B. Are there any prerequisite courses for students taking these advanced mathematics 
programs/tracks? 


Check one circle only. 
© Yes 


© No 


If Yes... 
Please explain: 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - Advanced Mathematics Curriculum 


iat Advanced Mathematics Curriculum 


3. A. Summarize the mathematics curriculum that was in effect for the students assessed in TIMSS 
Advanced 2015. (750 words) 


If applicable, please reference your country’s curricular documents. 


B. In what year was the advanced mathematics curriculum introduced? 


Examples of information ted for TIMSS Advanced 2008 can be found in the second column of Exhibit 1.3 on 33 of the 2008 
report. Click here to view 


Comments: 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015-Englisn (Continued) 
You are logged in as: 9911 Logou: 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - Advanced Mathematics Curriculum 


C. Is the advanced mathematics curriculum currently being revised? 


Examples of information reported for TIMSS Advanced 2008 can be found in the third column of Exhibit 1.3 on 33 of the 2008 
report. Click here to view 


Check one circle only. 


If Yes... 
Please explain: 


If No... 
Comments: 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire - Mathematics — Instructional Matcrials and Usc of Technology 


am Instructional Materials and Use of Technology 


4. Is there a process for approving the advanced mathematics instructional materials? 
Check one circle only. 
© Yes 


() No 


If Yes... 
Please describe the process, and what materials (e.g., textbooks, workbooks, online materials) must be 
approved through this process: 


{Previous 4/13 Table of Contents | Next 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - Instructional Materials and Use of Technology 


5. A. Does the curriculum contain statements/policies about the use of technology (e.g., computers, 
tablets, calculators) in advanced mathematics instruction? 


Check one circle only. 
Yes 


No 


if Yes... 
What are the statements/policies? 


Comments: 


B. Does the curriculum contain statements/policies about student use of technological aids (e.g., 
computers, tablets, calculators) in advanced mathematics tests or examinations? 


Check one circle only. 


Yes 


No 


If Yes... 
What are the statements/policies? 


Comments: 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - Examinations 


xz Examinations 


6. A. Does an educational authority in your country (e.g., National Ministry of Education) administer 
examinations to students in these advanced mathematics programsitracks that have consequences for 
individual students, such as entry to a university? 


Check one circle only. 


If Yes.... 
B. Please describe the secondary school grades at which the exams are given to students in each of 
these programs/tracks and the purpose of each exam. 


Examples of information reported for TIMSS Advanced 2008 can be found in the third and fifth columns of Exhibit 1.6 on eS 36-39 
of the 2008 report. Click here to view 


C. What is the nature and format of the examinations, and do they have an oral component? 


Examples of information reported for TIMSS Advanced 2008 can be found in the fourth column of Exhibit 1.6 on es 38-39 of the 
2008 report. Click here to view 


D. Additional comments on the examination system 


Examples of information reported for TIMSS Advanced 2008 can be found in the sixth column of Exhibit 1.6 on pages 38-39 of the 
2008 report. Click here to view 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9811 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - Advanced Mathematics Topics Covered 


tl Advanced Mathematics Topics Covered 


7. According to the curriculum, should the students in the advanced mathematics programs/tracks 
being assessed by TIMSS Advanced have been taught each of the following topics by the end of the 


year (in the current course or before)? 
If part of a topic does not apply [e.g., logarithmic expressions in par A topic (a)], please explain in the comment field. 


Check one circle for each line. 


A. Algebra 


a) Operations with exponential, logarithmic, polynomial, rational, and radical 
expressions 


b) Operations with compex numbers 

c) Evaluating algebraic expressions (e.g., exponential, logarithmic, polynomial, 
rational, and radical) 
The nth term of arithmetic and geometric sequences and the sums of finite 
and infinite series 
Linear, simultaneous, and quadratic equations and inequalities; radical equations, 
logarithmic, and exponential equatons 
Slopes, y-axis intercepts, and points of intersection of straight lines 
Equivalent representations of functions, including composite functions, as ordered pairs, 
tables, graphs, formulas, or words 

h) Properties of functions incuding domain and range 


Comments: 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9811 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - Advanced Mathematics Topics Covered 
7. (continued) 
According to the curriculum, should the students in the advanced mathematics programs/tracks being 


assessed by TIMSS Advanced have been taught each of the following topics by the end of the year (in 
the current course or before)? 


If part of a topic does not apply [e.g., logarithmic expressions in part A topic (a)], please explain in the comment field. 


Check one circle for each line. 


B. Calculus 
a) Limits of functions 


b) Conditions for continuty and cifferentiability of functions 


¢) Differentiation of functions (including polynomial, exponential, logarithmic, trigonometric, 
rational, and radical functions); differentiation of products, quotients, and composite 
functions 


Using derivatives to solve problems (e.g., in optimization and rates of change) 
Using first and second defivatives to determine slope and local extrema 

of functions 

Using derivatives to determine points of inflection of functions 

Integrating functions (including polynomial, exponential, trigonometnc, 


and rational functions}; evaluating definite integrals, including calculation 
of areas 


Comments: 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 
TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - Advanced Mathematics Topics Covered 


7. (continued) 

According to the curriculum, should the students in the advanced mathematics programs/tracks being 
assessed by TIMSS Advanced have been taught each of the following topics by the end of the year (in 
the current course or before)? 


If part of a topic does not apply fe.g., logarithmic expressions in part A topic (a)j, please explain in the comment field. 


Check one circie for each line. 


C. Geometry 
a) Properties of geometric figures in two and three dimensions 


b) Properties of vectors and their sums and défferences 
c) Trigonometric properties of triangles (sine, cosine, and tangent) 
d) Trigonometric functions and their graphs 


Comments: 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged In as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - Advanced Mathematics Topics Covered 


8. How is the implementation of the advanced mathematics curriculum evaluated? 


Check one circle for each line. 


a) Visits by inspectors 

b) Research programs 

c) School self-evaluation 

d) National or regional examinations 


e) Other 
Please specify beow: 


Comments: 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - Recruitment to TIMSS Advanced Programs/Tracks 


BS Recruitment to TIMSS Advanced Programs/Tracks 


9. A. Does your country sponsor national programs to encourage students to study advanced 
mathematics? 


Check one circle only. 


= Yes 
, No 


If Yes... 


B. Does your country implement any of the following programs to promote the study of advanced 
mathematics? 


Check one circle for each line. 


a) School partnerships with industry 
6) School collaborations with universities 
c) Contests/competitions in advanced mathematics 
d) Other 
Please specify: 


If applicable, please describe the programs implemented in your country to promote the study of advanced mathematics: 


Previous 11/13 Table of Cortents Next 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide D-369 


Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - Advanced Mathematics Teachers 


4 Advanced Mathematics Teachers 


10. Describe the national requirements for being a teacher of the advanced mathematics 
programsitracks being assessed in TIMSS Advanced. 


12N3 Table of Contents 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics - Advanced Mathematics Teachers 
11. Does your country experience any difficulties recruiting or retaining advanced mathematics teachers 
of students ? 
Check one circle only. 


Yes 


, No 


If Yes... 
Comments: 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9811 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Mathematics 


This completes the TIMSS Advanced 2015 Curriculum Questionnaire - Advanced Mathematics Module 


To submit your completec questionnaire, please click the Finish button. 
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Exhibit D-15. TIMSS Advanced 2015 Math Curriculum Questionnaire—Continued 


TIMSS bomen’ 
AUS 
Advanced 

Mathematics 


International Association 
for the Evaluation of 
Educational Achievement 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire 


TEA, 


© IEA, 2015 


TIMSS Advanced 2015 
Curriculum Questionnaire— 
Physics 


Physics 


_ TIMSS & PIRLS 
International Study Center 


h Sehocl of Education, Basten College 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSSA2015MS_OCQ - English 
You are not logged in. 


IEA, 
TIMSS TEAS 


2015 amas 
Welcome to the IEA - DPC SurveySystem 


TIMSS Advanced 2015 
Curriculum Questionnaire 


Please enter your user ID and password (Checksum). 


User ID: 


Password: 


Login 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics 


The TIMSS Advanced 2015 Curriculum Questionnaires are designed to collect basic information about the structure of the education 
system as well as the organization, content, and implementation of the advanced mathematics and physics curricula in gach country. 
There are separate questionnaires for Advanced Mathematics and Physics. 


The questionnaires should be completed by the National Research Coordinators, drawing on the expertise of curriculum specialists 
and educators. Please submit the questionnaires no later than August 31, 2015. 


To begin this questionnaire, please click on the “Next” button. When navigating through the questionnaire, make sure to confirm your 
responses by clicking on the “Next” or “Previous” button. To go to a particular section or item, please click on the corresponding link in 
the “Table of Contents”. 


If you have any questions about the content of this questionnaire please contact the TIMSS & PIRLS International Study Center at 
Boston College: timss@bc.edu 


If you have any technical questions on how to complete this questionnaire, please contact the IEA Data Processing & Research Center 
(DPC): timss@iea-dpc.de 


Table of Contents Next 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2016 Curriculum Questionnaire — Physics About the Physics Programs (Tracks) 


ml About the Physics Programs (Tracks) 


This questionnaire refers to the national physics curriculum that was in effect for the students assessed in TIMSS Advanced 2015—the 
curriculum that covers physics instruction for the majority of students in these programs or tracks. If you do not have a nationai 
curriculum, please summarize for your state or provincial curricula. 


1. A. Describe the physics programs/tracks assessed by TIMSS Advanced 2015. How do the 
programs/tracks fit into the overall curriculum from the first grade through the final year? How do they 
relate with programs at the university level, if at all (e.g., is participation a prerequisite for studying 
certain fields such as engineering or medicine)? 


Examples of information reported for TIMSS Advanced 2008 can be found in the second column of Exhibit 7.1 on pages 220-221 of 
the 2008 report. Click here to view 


B. How many years are students in these programs/tracks, and at which grade do they start? 


Examples of information reported for TIMSS Advanced 2008 can be found in the third column of Exhibit 7.1 on pages 220-221 of the 
2008 report. Click here to view 


C. What is the total amount of class time in physics for the students in the physics programs/tracks? 


Examples of information reported for TIMSS Advanced 2008 can be found in the fourth column of Exhibit 7.1 on pages 220-221 of the 
2008 report. Click here to view 


hours per year (1 hour = 60 minutes) 


Comments: 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 
TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics - Criteria for Admission 


fa Criteria for Admission 


2. A. What are the criteria for admission to these physics programs/tracks? 
Exa 


mples of information reported for TIMSS Advanced 2008 can be found in the fifth column of Exhibit 7.7 on eS 220-221 of the 
2008 report. Click here to view 


B. Are there any prerequisite courses for students taking these physics programs/tracks? 


Check one circle only. 


, Yes 
, No 


If Yes... 
Please explain: 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics - Physics Curriculum 


_ Physics Curriculum 


3. A. Summarize the physics curriculum that was in effect for the students assessed in TIMSS Advanced 
2015. (750 words) 


If applicable, please reference your country’s curricular documents, 


B. In what year was the physics curriculum introduced? 


Ex s of information reported for TIMSS Advanced 2008 can be found in the second column of Exhibit 7.3 on 226 of the 
2008 report. Click here to view 


Comments: 


(Continued on Next Page) 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 


You are logged in as: 9911 Logout (Cor itinued) 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics - Physics Curriculum 


C. Is the physics curriculum currently being revised? 


Examples of information reported for TIMSS Advanced 2008 can be found in the third column of Exhibit 7.3 on page 226 of the 2008 
report. Click here to view 


Check one circle only. 


C) Yes 
No 


If Yes... 
Please explain: 


If No... 
Comments: 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics - Instructional Materials and Use of Technology 


zx Instructional Materials and Use of Technology 


4. ls there a process for approving the physics instructional materials? 
Check one circle only. 

Yes 

No 
If Yes... 


Please describe the process, and what materials (e.g., textbooks, workbooks, online materials) must be 
approved through this process: 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics - Instructional Materials and Use of Technology 
5. A. Does the curriculum contain statements/policies about the use of technology (e.g., computers, 
tablets, calculators) in physics instruction? 
Check one circle only. 


© Yes 


If Yes... 
What are the statements/policies? 


Comments: 


B. Does the curriculum contain statements/policies about student use of technological aids (e.g., 
computers, tablets, calculators) in physics tests or examinations? 


Check one circle only. 
O Yes 
©) No 


if Yes... 
What are the statements/policies? 


Comments: 


ss 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics - Examinations 


_ Examinations 


6. A. Does an educational authority in your country (e.g., National Ministry of Education) administer 
examinations to students in these physics programsitracks that have consequences for individual 
students, such as entry to a university? 


Check one circle only. 


, Yes 
, No 


If Yes.... 
B. Please describe the secondary school grades at which the exams are given to students in each of 
these programs/tracks and the purpose of each exam. 


Examples of information reported for TIMSS Advanced 2008 can be found in the third and fifth columns of Exhibit 7.6 on es 230- 
231 of the 2008 report. Click here to view 


C. What is the nature and format of the examinations, and do they have an oral component? 


Examples of information reported for TIMSS Advanced 2008 can be found in the fourth column of Exhibit 7.6 on eS 230-231 of the 
2008 report. Click here to view 


D. Additional comments on the examination system 


Examples of information reported for TIMSS Advanced 2008 can be found in the sixth column of Exhibit 7.6 on iS 230-231 of the 
2008 report. Click here to view 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire - Physics Pnysics Topics Covercd 


| Physics Topics Covered 


7. According to the curriculum, should the students in the physics programs/tracks being assessed by 
TIMSS Advanced have been taught each of the following topics by the end of the year (in the current 
course or before)? 


If part of a topic does not apply fe.q., expansion of solids and liquids in relation to temperature change in part A topic (i), please 
explain in the comment field. 


Check one circle for each line. 


A. Mechanics and Thermodynamics 
a) Applying Newton's laws and laws of motion 


b) Forces, including frictional force, acting on a body 


c) Forces acting on a body moving in a circular path; the body's centripetal acceleration, 
speed, and circling time 


d) The law of gravitation in relation to the movement of celestial objects 

e) Kinetic and potential energy; conservation of mechanical energy 

f) The law of conservation of momentum; elastic and inelastic collisions 

g) The first law of thermodynamics 

h) Heat transfer and specific heat caoacities 

i) The law of ideal gases; expansion of solids and liquids in relation to temperature change 


Comments: 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 
TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics - Physics Topics Covered 


7. (continued) 
According to the curriculum, should the students in the physics programs/tracks being assessed by 


TIMSS Advanced have been taught each of the following topics by the end of the year (in the current 
course or before)? 


If part of a topic does not apply fe.g., expansion of solids and liquids in relation to temperature change in part A topic (i)], please 
explain in the comment field. 


Check one circle for each line. 


B. Electricity and Magnetism Yes 
a) Electrostatic attraction or repulsion between isolated charged particles—Coulomb’s law C) 


b) Charged particles in an electric field 
c) Electrical circuits; using Ohm's law and Joule’s law 
d) Charged particles in a magnetic field 


e) Relationship between magnetism and electricity; magnetic fields around electric 
conductors; electromagnetic induction 


f) Faraday’s and Lenz's laws of induction 


Comments: 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire - Physics Physics Topics Covercd 


7. (continued) 

According to the curriculum, should the students in the physics programs/tracks being assessed by 
TIMSS Advanced have been taught each of the following topics by the end of the year (in the current 
course or before)? 


If part of a topic does not apply fe.g., expansion of solids and liquids in relation to temperature change in part A topic (i)}, please 
explain in the comment field. 


Check one circle for each line. 


C. Wave Phenomena and Atomic/Nuclear Physics 

a) Mechanical waves; the relationship between speed, frequency, and wavelength 

b) Electromagnetic radiation; wavelength and frequency of varicus types of waves 
(radio, infrared, visible light, x-rays, gamma rays) 

c) Thermal radiation, temperature, and wavelength 

d) Reflection, refraction, interference, and diffraction 

e) The structure of the atom and its nucleus; atomic number and atomic mass; 
electromagnetic emission and absorption and the behavior of electrons 


f) Wave-oarticle duality and the photoelectric effect; types of nuclear reactions 
and their role in nature (e.g., in stars) and society; radioactive isotopes 


g) Mass-energy equivalence in nuclear reactions and particle transformations 


Comments: 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics - Physics Topics Covered 


8. How is the implementation of the physics curriculum evaluated? 


Check one circle for each line. 


a) Visits by inspectors 

b) Research programs 

c) School self-evaluation 

d) National or regional examinations 


e) Other 
Please specify below: 


Comments: 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics - Recruitment to TIMSS Advanced Programs/Tracks 


a Recruitment to TIMSS Advanced Programs/Tracks 


9. A. Does your country sponsor national programs to encourage students to study physics? 
Check one circle only. 


>) Yes 
>) No 


If Yes... 
B. Does your country implement any of the following programs to promote the study of physics? 


Check one circle for each line. 
No 
a) School partnerships with industry . 
6) School collaborations with universities 
c) Contests/competitions in physics 
d) Other 
Please specify: 


if applicabe, please describe the programs implemented in your country to promote the stuay of physics: 


| _ Previous 11/13 Table of Contents | Next 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics - Physics Teachers 


|| Physics Teachers 


10. Describe the national requirements for being a teacher of the physics programs/tracks being 
assessed in TIMSS Advanced. 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 
TIMSS Advanced - 2015 - English 


You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics - Physics Teachers 


11. Does your country experience any difficulties recruiting or retaining physics teachers of students at 
the end of upper secondary school? 


Check one circle only. 
, Yes 


~) No 


If Yes... 
Comments: 


Previous 


13/13 Table of Contents 


Next 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


TIMSS Advanced - 2015 - English 
You are logged in as: 9911 Logout 


TIMSS Advanced 2015 Curriculum Questionnaire — Physics 


This completes the TIMSS Advanced 2015 Curriculum Questionnaire - Physics Module 


To submit your completed questionnaire, please click the Finish button. 


Previous Table of Contents Finish 
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Exhibit D-16. TIMSS Advanced 2015 Physics Curriculum Questionnaire—Continued 


COLLEGE 
timss.bc.edu 


TIMSS 


2015 


Physics 


© IEA, 2015 

International Association 
for the Evaluation of 
Educational Achievement 
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E-1 


Exhibit E-1. 


TIMSS 2015 Grade 4 School Questionnaire 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
ScQ-02 What is the total enrollment of ScQ-02 What is the total enrollment of fourth- 
<fourth grade> students in your grade students in your school as of 
school as of <first day of month March 1, 2015? 
TIMSS testing begins, 2015>? students 
Write in the number. Write in the number 
students 
ScQ-04 Around the 1st of October 2014, what 
percentage of students at this school 
were eligible to receive free or 
reduced-price lunches through the 
National School Lunch Program? 
percentage of students 
Write in the number. 

ScQ-04 Approximately what percentage of ScQ-05 Approximately what percentage of 
students in your school have students in your school have English 
<language of test> as their native as their native language? 
language? Fill in one circle only. 

Check one circle only. 1. More than 90% 

1. More than 90% 2. 76 to 90% 

2. 76 to 90% 3. 51 to 75% 

3. 51 to 75% 4. 26 to 50% 

4. 26 to 50% 5. 25% or less 

5. 25% or less 

ScQ-06 Of the students currently enrolled in 
your school, what percentage has 
been identified as limited-English 
proficient (LEP)/English language 
learners (ELL)? 
Fill in one circle only. 
1.0% 
2. 1-5% 
3. 6-10% 
4. 11-25% 
5. 26-50% 
6. 51-75% 
7. 16-90% 
8. Over 90% 
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Exhibit E-1. 


TIMSS 2015 Grade 4 School Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International 
Item Number 


Item 


National 
Item Number 


Item 


Recoding 
instructions 


ScQ-07 


What type of school is this? 

Fill in one circle only. 

1. Regular public school 

2. A regular public school with a magnet 
program 

3. A magnet school or school with a 
special program emphasis (e.g., 
Montessori, science/math school, 
performing arts school, talented/gifted 
school, foreign language immersion 
school) 

4. Special education: a school that 
primarily serves students with 
disabilities 

5. Alternative: a school designed to 
address the needs of students, typically 
at risk of educational failure, which 
cannot be met in regular schools 

6. Vocational 

7. Charter school 

8. Private (independent) 

9. Private (religiously affiliated) 

10. Other 


ScQ-09 


Which best characterizes the average 
income level of the school’s 
immediate area? 

Fill in one circle only. 

1. High 

2. Medium 

3. Low 


ScQ-08A 


For the <fourth grade> students in 
your school: 
How many days per year is your 
school open for instruction? 
Write in the number. 

days 


ScQ-12A 


For the fourth-grade students in your 
school: 
How many days per year is your 
school open for instruction? 

days 
Write in the number. 


ScQ-08B 


What is the total instructional time, 
excluding breaks, in a typical day? 
Write in the number of minutes per day. 
Please convert the number of hours into 
minutes. 

minutes 


ScQ-12B 


What is the total instructional time, 

excluding breaks, in a typical day? 
hours minutes 

Write in the number of hours and 


minutes per day. 


1hr-->60min 
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Exhibit E-1. 


TIMSS 2015 Grade 4 School Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
ScQ-09A Does your school provide a place ScQ-13A Does your school provide a place 

where students can work on their where students can work on their 

schoolwork before or after school? schoolwork before or after school? 

Check one circle only. Fill in one circle only. 

1.Yes 1.Yes 

2. No 2. No 

(If No, go to #10) (If No, go to question 14) 

ScQ-10 As a general school policy, is student | ScQ-14 As a general school policy, is student 
achievement used to assign <fourth achievement used to assign fourth- 
grade> students to classes (e.g., grade students to classes (e.g., 
streaming, tracking, setting)? streaming, tracking, setting)? 
Check one circle for each line. Fill in only one circle for each row. 

1. Yes 1. Yes 
2. No 2. No 

ScQ-11 How many computers (including ScQ-15 How many computers (including 
tablets) does your school have for tablets) does your school have for 
use by <fourth grade> students? use by fourth-grade students? 
Write in the number. computers 

computers Write in the number. 

ScQ-12A Does your school have a science ScQ-16A Does your school have a science 
laboratory that can be used by laboratory that can be used by 
<fourth grade> students? fourth-grade students? 

Check one circle only. Fill in one circle only. 
1. Yes 1. Yes 
2. No 2. No 

ScQ-13 Does your school have a school ScQ-17 Does your school have a school 

library? library? 

Check one circle only. Fill in one circle only. 

1. Yes 1. Yes 

2. No 2. No 

(If No, go to #14) (If No, go to question 18) 

ScQ-16 To what degree is each of the ScQ-20 To what degree is each of the 
following a problem among <fourth following a problem among fourth- 
grade> students in your school? grade students in your school? 
Check one circle for each line. Fill in only one circle for each row. 

1. Not a problem 1. Not a problem 
2. Minor problem 2. Minor problem 
3. Moderate problem 3. Moderate problem 
4. Serious problem 4. Serious problem 
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Exhibit E-1. _TIMSS 2015 Grade 4 School Questionnaire—Continued 


Questions that Require National Adaptations 
2015 International Version 2015 U.S. Adapted Version 
International National Recoding 
Item Number | Item Item Number | Item instructions 
ScQ-22 In your school, are any of the 
following used to evaluate the 
practice of fourth-grade teachers? 
Fill in only one circle for each row. 
1. Yes 
2. No 
ScQ-22a Observations by the principal or senior 
staff 
ScQ-22b Observations by inspectors or other 
persons external to the school 
ScQ-22c Student achievement 
ScQ-22d Teacher peer review 
ScQ-18 About how many of the students in = |ScQ-23 About how many of the students in 
your school can do the following your school can do the following 
when they begin the <first grade> of when they begin the first grade of 
primary/elementary school? primary/elementary school? 
Check one circle for each line. Fill in only one circle for each row. 
1. Less than 25% 1. Less than 25% 
2. 25-50% 2. 25-50% 
3. 51-75% 3. 51-75% 
4. More than 75% 4. More than 75% 
ScQ-19 By the end of this school year, how | ScQ-24 By the end of this school year, how 
many years will you have been a many years altogether will you have 
principal altogether? been a principal? 
Please round to the nearest whole years 
number. Please round to the nearest whole 
years number. 
ScQ-21 What is the highest level of formal ScQ-26 What is the highest level of formal Nat -> Int 
education you have completed? education you have completed? 1->1 
Check one circle only. Fill in one circle only. 2->2 
1. Did not complete <Bachelor's or 1. Did not complete Bachelor's degree |3->3 
equivalent level—ISCED Level 6> (4-year college program) 4->4 
2. <Bachelor's or equivalent level— 2. Bachelor's degree (4-year college 
ISCED Level 6> program) 
3. <Master's or equivalent level—ISCED 3. Master's degree or professional 
Level 7> degree (MD, DDS, lawyer, minister) 
4. <Doctor or equivalent level—ISCED 4. Doctorate (Ph.D., or Ed.D.) 
Level 8> 
ScQ-22a <Master's or equivalent level—ISCED  |ScQ-27a Master's degree or professional degree 
Level 7> (MD, DDS, lawyer, minister) 
ScQ-22b <Doctor or equivalent level—ISCED ScQ-27b Doctorate (Ph.D., or Ed.D.) 
Level 8> 
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Exhibit E-2. TIMSS 2015 Grade 4 Teacher Questionnaire 


Questions that Require National Adaptations 


2015 International Version 2015 U.S. Adapted Version 
International National Recoding 
Item Number | Item Item Number | Item instructions 
TQG-01 What year did you start teaching? 
Please write in a year. 
TQG-01 By the end of this school year, how = |TQG-02 At the end of this school year, how 
many years will you have been many years will you have taught 
teaching altogether? altogether? 
Please round to the nearest whole years 
number. Please round to the nearest whole 
years number. 
TQG-04 What is the highest level of formal TQG-05 What is the highest level of formal Nat -> Int 
education you have completed? education you have completed? 1->1 
Check one circle only. Fill in one circle only. 2->2 
1. Did not complete <Upper secondary 1. Did not complete high school 3->4 
education—ISCED Level 3> 2. High school graduate 4->5 
2. <Upper secondary education— (If you have not completed more than 5->6 
ISCED Level 3> high school, go to question 7) 6->7 
(If you have not completed <post- 3. Associate's degree (2-year college International 
secondary or tertiary education>, program) Category 4 
go to #G6) 4. Bachelor's degree (4-year college (ISCED Level 
3. <Post-secondary, non-tertiary program) 4) is not 
education—ISCED Level 4> 5. Master's degree or professional administered 
4. <Short-cycle tertiary education— degree (MD, DDS, lawyer, minister) 
ISCED Level 5> 6. Doctorate (Ph.D., or Ed.D.) 
5. <Bachelor’s or equivalent level— 
ISCED Level 6> 
6. <Master’s or equivalent level— 
ISCED Level 7> 
7. <Doctor or equivalent level— 
ISCED Level 8> 
TQG-05A During your <post-secondary> TQG-06A During your college or university 
education, what was your major or education, what was your major or 
main area(s) of study? main area(s) of study? 
Check one circle for each line. Fill in only one circle for each row. 
1. Yes 1. Yes 
2. No 2. No 
TQG-05Ae <language of test> TQG-06Ae English 
TQG-05B If your major or main area of study TQG-06B If your major or main area of study 
was education, did you have a was education, did you have a 
<specialization> in any of the specialization in any of the 
following? following? 
Check one circle for each line. Fill in only one circle for each row. 
1. Yes 1. Yes 
2. No 2. No 
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Exhibit E-2. 


TIMSS 2015 Grade 4 Teacher Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQG-12B How many of the students in#G12A |TQG-13B How many of the students in 
are in <fourth grade>? question 13A are in fourth grade? 
Write in the number. fourth-grade students 
<fourth grade> Write in the number. 
students 

TQG-13 How many <fourth grade> students |TQG-14 How many fourth-grade students 
experience difficulties understanding experience difficulties understanding 
spoken <language of test>? spoken English? 

Write in the number. students in this class 
students in this class Write in the number. 

TQM-05A Do the students in this class have TQM-21A Do the students in this class have 
computers (including tablets) computers (including tablets) 
available to use during their available to use during their 
mathematics lessons? mathematics lessons? 

Check one circle only. Fill in one circle only. 

1. Yes 1. Yes 

2. No 2. No 

(If No, go to #M6) (If No, go to question 22) 

TQM-06 The following list includes the main | TQM-22 The following list includes the main 
topics addressed by the TIMSS topics addressed by the TIMSS 
mathematics test. Choose the mathematics test. Choose the 
response that best describes when response that best describes when 
the students in this class have been the students in this class have been 
taught each topic. If a topic was in taught each topic. If a topic was in 
the curriculum before the <fourth the curriculum before the fourth 
grade>, please choose “Mostly grade, please choose “Mostly taught 
taught before this year.” If a topic before this year.” If a topic was 
was taught half this year but not yet taught half this year but not yet 
completed, please choose “Mostly completed, please choose “Mostly 
taught this year.” If a topic is not in taught this year.” If a topic is not in 
the curriculum, please choose “Not the curriculum, please choose “Not 
yet taught or just introduced.” yet taught or just introduced.” 

Check one circle for each line. Fill in only one circle for each row. 
1. Mostly taught before this year 1. Mostly taught before this year 
2. Mostly taught this year 2. Mostly taught this year 

3. Not yet taught or just introduced 3. Not yet taught or just introduced 

TQM-07A How often do you usually assign TQM-23A How often do you usually assign 
mathematics homework to the mathematics homework to the 
students in this class? students in this class? 

Check one circle only. Fill in one circle only. 
1. | do not assign mathematics 1. | do not assign mathematics 
homework homework 
(Go to #M8) (Go to question 24) 
2. Less than once a week 2. Less than once a week 
3. 1 or 2 times a week 3. 1 or 2 times a week 
4. 3 or 4 times a week 4. 3 or 4 times a week 
5. Every day 5. Every day 
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Exhibit E-2. 


TIMSS 2015 Grade 4 Teacher Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQM-08c National or regional achievement tests | TQM-24c State or district achievement tests 
TQM-10 In the past two years, how many TQM-26 In the past two years, how many 
hours in total have you spent in hours in total have you spent in 
formal <in-service/professional formal in-service/professional 
development> (e.g., workshops, development (e.g., workshops, 
seminars, etc.) for mathematics? seminars) for mathematics? 
Check one circle only. Fill in one circle only. 
1. None 1. None 
2. Less than 6 hours 2. Less than 6 hours 
3. 6-15 hours 3. 6-15 hours 
4. 16-35 hours 4. 16-35 hours 
5. More than 35 hours 5. More than 35 hours 
TQM-11 How well prepared do you feel you TQM-27 How well prepared do you feel you 
are to teach the following are to teach the following 
mathematics topics? mathematics topics? 
If a topic is not in the <fourth grade> If a topic is not in the fourth-grade 
curriculum or you are not curriculum or you are not 
responsible for teaching this topic, responsible for teaching this topic, 
please choose “Not applicable.” please choose “Not applicable.” 
Check one circle for each line. Fill in only one circle for each row. 
1. Not applicable 1. Not applicable 
2. Very well prepared 2. Very well prepared 
3. Somewhat prepared 3. Somewhat prepared 
4. Not well prepared 4. Not well prepared 
TQS-04A Do the students in this class have TQS-31A Do the students in this class have 
computers (including tablets) computers (including tablets) 
available to use during their science available to use during their science 
lessons? lessons? 
Check one circle only. Fill in one circle only. 
1. Yes 1. Yes 
2. No 2. No 
(If No, go to #S5) (If No, go to question 32) 
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Exhibit E-2. TIMSS 2015 Grade 4 Teacher Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQS-05 The following list includes the main |TQS-32 The following list includes the main 
topics addressed by the TIMSS topics addressed by the TIMSS 
science test. Choose the response science test. Choose the response 
that best describes when the that best describes when the 
students in this class have been students in this class have been 
taught each topic. If a topic was in taught each topic. If a topic was in 
the curriculum before the <fourth the curriculum before the fourth 
grade>, please choose “Mostly grade, please choose “Mostly taught 
taught before this year.” If a topic before this year.” If a topic was 
was taught half this year but not yet taught half this year but not yet 
completed, please choose “Mostly completed, please choose “Mostly 
taught this year.” If a topic is not in taught this year.” If a topic is not in 
the curriculum, please choose “Not the curriculum, please choose “Not 
yet taught or just introduced.” yet taught or just introduced.” 
Check one circle for each line. Fill in only one circle for each row. 
1. Mostly taught before this year 1. Mostly taught before this year 
2. Mostly taught this year 2. Mostly taught this year 
3. Not yet taught or just introduced 3. Not yet taught or just introduced 
TQS-06A How often do you usually assign TQS-33A How often do you usually assign 
science homework to the students in science homework to the students in 
this class? this class? 
Check one circle only. Fill in one circle only. 
1. | do not assign science homework 1. | do not assign science homework 
(Go to #S7) (Go to question 34) 
2. Less than once a week 2. Less than once a week 
3. 1 or 2 times a week 3. 1 or 2 times a week 
4. 3 or 4 times a week 4. 3 or 4 times a week 
5. Every day 5. Every day 
TQS-07c National or regional achievement tests | TQS-34c State or district achievement tests 
TQS-09 In the past two years, how many TQS-36 In the past two years, how many 
hours in total have you spent in hours in total have you spent in 
formal <in-service/professional formal in-service/professional 
development> (e.g., workshops, development (e.g., workshops, 
seminars, etc.) for science? seminars, etc.) for science? 
Check one circle only. Fill in one circle only. 
1. None 1. None 
2. Less than 6 hours 2. Less than 6 hours 
3. 6-15 hours 3. 6-15 hours 
4. 16-35 hours 4. 16-35 hours 
5. More than 35 hours 5. More than 35 hours 
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Exhibit E-2. 


TIMSS 2015 Grade 4 Teacher Questionnaire —Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQS-10 How well prepared do you feel you TQS-37 How well prepared do you feel you 

are to teach the following science are to teach the following science 

topics? topics? 

If a topic is not in the <fourth grade> If a topic is not in the fourth-grade 

curriculum or you are not curriculum or you are not 

responsible for teaching this topic, responsible for teaching this topic, 

please choose “Not applicable.” please choose “Not applicable.” 

Check one circle for each line. Fill in only one circle for each row. 

1. Not applicable 1. Not applicable 

2. Very well prepared 2. Very well prepared 

3. Somewhat prepared 3. Somewhat prepared 

4. Not well prepared 4. Not well prepared 
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Exhibit E-3. 


TIMSS 2015 Grade 4 Student Questionnaire 


Questions that Require National Adaptations 
2015 International Version 2015 U.S. Adapted Version 
International National Recoding 
Item Number | Item Item Number | Item instructions 
SQG-01B Are you Hispanic or Latino? 
Fill in one oval only. 
1. Yes, | am Hispanic or Latino 
2. No, | am not Hispanic or Latino 
SQG-01C Which of the following best 
describes you? 
Fill in ovals for all that apply. 
1. White 
2. Black or African American 
3. Asian 
4. American Indian or Alaska Native 
5. Native Hawaiian or other Pacific 
Islander 
SQG-03 How often do you speak <language /|SQG-03A How often do you speak English at 
of test> at home? home? 
Fill one circle only. Fill in one oval only. 
1. | always speak <language of test> at 1. | always speak English at home 
home If Always, please go to question 4. 
2. | almost always speak <language of 2. | almost always speak English at 
test> at home home 
3. | sometimes speak <language of 3. | sometimes speak English and 
test> and sometimes speak another sometimes speak another language at 
language at home home 
4. | never speak <language of test> at 4. | never speak English at home 
home If Almost always, Sometimes, Never, 
please go to question 3B. 
SQG-03B What language do you speak at home 
(other than English)? 
Fill in one oval only. 
1. Spanish 
2. Other Please specify 
SQG-05g A gaming system (e.g., PlayStation®, SQG-05g A gaming system (e.g., PlayStation, Wii, 
Wii®, XBox®) Xbox) 
SQG-05h <country-specific indicator of wealth> SQG-05h VCR, DVD, or Blu-ray player 
SQG-06A Was your mother (or stepmother or |SQG-06A Was your mother (or stepmother or 
female guardian) born in <country>? female legal guardian) born in the 
Fill one circle only. United States? (“United States” 
1. Yes includes the 50 states, its territories, 
2. No the District of Columbia, and U.S. 
3. | don't know military bases abroad) 
Fill in one oval only. 
1. Yes 
2. No 
3. | don't know 
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Exhibit E-3. 


TIMSS 2015 Grade 4 Student Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International 
Item Number 


Item 


National 
Item Number 


Item 


Recoding 
instructions 


SQG-06B 


Was your father (or stepfather or 
male guardian) born in <country>? 
Fill one circle only. 

1. Yes 

2. No 

3. | don't know 


SQG-06B 


Was your father (or stepfather or 
male legal guardian) born in the 
United States? 

Fill in one oval only. 

1. Yes 

2. No 

3. | don't know 


SQG-07 


Were you born in <country>? 
Fill one circle only. 

1. Yes 

2. No 


SQG-07 


Were you born in the United States? 
Fill in one oval only. 

1. Yes 

2. No 


SQG-08 


The following questions ask about 
activities you do outside of school. 
Fill in only one oval for each row. 

1. Yes 

2. No 


SQG-08a 


Do you play on a sports team outside of 
school? 


SQG-08b 


Do you often play a musical instrument 
outside of school? 


SQG-08c 


Are you studying something in a class 
outside of school? 


SQG-08d 


SQG-09 


Do you belong to a club outside of 
school (like Boy/Girl Scouts, 4-H, or 
Boys and Girls Club)? 

Are you preparing for or have you 
participated in a science club, a 
science fair, or a science 
competition? 

Fill in one oval only. 

1. Yes 

2. No 


SQG-10 


Have you ever repeated a grade in 
elementary school? 

Fill in one oval only. 

1. Yes 

2. No 


SQG-11B 


How many days were you absent 
from school in the last month? 
Fill in one oval only. 

1. None 

2. 1 or 2 days 

3. 3 or 4 days 

4. 5 to 10 days 

5. More than 10 days 
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Exhibit E-3. 


TIMSS 2015 Grade 4 Student Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International 
Item Number 


Item 


National 
Item Number 


Item 


Recoding 
instructions 


SQMS-22 


How hard was this test compared to 
most other tests you have taken this 
year in school? 

Fill in one oval only. 

1. Easier than other tests 

2. About as hard as other tests 

3. Harder than other tests 

4. Much harder than other tests 


SQMS-23 


How hard did you try on this test 
compared to how hard you tried on 
most other tests you have taken this 
year in school? 

Fill in one oval only. 

1. Not as hard as on other tests 

2. About as hard as on other tests 

3. Harder than on other tests 

4. Much harder than on other tests 


SQMS-24 


How important was it to you to do 
well on this test? 

Fill in one oval only. 

1. Not very important 

2. Somewhat important 

3. Important 

4. Very important 
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Exhibit E-4. 


TIMSS 2015 Grade 8 School Questionnaire 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
ScQ-01 What is the total enrollment of ScQ-01 What is the total enrollment of 

students in your school as of <first students in your school as of March 

day of month TIMSS testing begins, 1, 2015? 

2015>? students 

Write in the number. Write in the number. 

students 

ScQ-02 What is the total enrollment of ScQ-02 What is the total enrollment of 
<eighth grade> students in your eighth-grade students in your school 
school as of <first day of month as of March 1, 2015 
TIMSS testing begins, 2015>? students 
Write in the number. Write in the number. 

students 

ScQ-04 Around the 1st of October 2014, what 
percentage of students at this school 
were eligible to receive free or 
reduced-price lunches through the 
National School Lunch Program? 

percentage of students 

Write in the number. 

ScQ-04 Approximately what percentage of ScQ-05 Approximately what percentage of 
students in your school have students in your school have English 
<language of test> as their native as their native language? 
language? Fill in one circle only. 

Check one circle only. 1. More than 90% 

1. More than 90% 2. 76 to 90% 

2. 76 to 90% 3. 51 to 75% 

3. 51 to 75% 4. 26 to 50% 

4. 26 to 50% 5. 25% or less 

5. 25% or less 

ScQ-06 Of the students currently enrolled in 
your school, what percentage has 
been identified as limited-English 
proficient (LEP)/English language 
learners (ELL)? 
Fill in one circle only. 
1. 0% 
2. 1-5% 
3. 6-10% 
4. 11-25% 
5. 26-50% 
6. 51-75% 
7. 76-90% 
8. Over 90% 
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Exhibit E-4. 


TIMSS 2015 Grade 8 School Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International 
Item Number 


Item 


National 
Item Number 


Item 


Recoding 
instructions 


ScQ-07 


What type of school is this? 

Fill in one circle only. 

1. Regular public school 

2. A regular public school with a magnet 
program 

3. A magnet school or school with a 
special program emphasis (e.g., 
Montessori, science/math school, 
performing arts school, talented/gifted 
school, foreign language immersion 
school) 

4. Special education: a school that 
primarily serves students with 
disabilities 

5. Alternative: a school designed to 
address the needs of students, typically 
at risk of educational failure, which 
cannot be met in regular schools 

6. Vocational 

7. Charter school 

8. Private (independent) 

9. Private (religiously affiliated) 

10. Other 


ScQ-09 


Which best characterizes the average 
income level of the school’s 
immediate area? 

Fill in one circle only. 

1. High 

2. Medium 

3. Low 


ScQ-07A 


For the <eighth grade> students in 
your school: 
How many days per year is your 
school open for instruction? 
Write in the number. 

days 


ScQ-11A 


For the eighth-grade students in your 
school: 
How many days per year is your 
school open for instruction? 

days 
Write in the number. 


ScQ-07B 


What is the total instructional time, 
excluding breaks, in a typical day? 
Write in the number of minutes per day. 
Please convert the number of hours into 
minutes. 

minutes 


ScQ-11B 


What is the total instructional time, 

excluding breaks, in a typical day? 
hours minutes 

Write in the number of hours and 


minutes per day. 
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Exhibit E-4. 


TIMSS 2015 Grade 8 School Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
ScQ-08A Does your school provide a place ScQ-12A Does your school provide a place 

where students can work on their where students can work on their 

schoolwork before or after school? schoolwork before or after school? 

Check one circle only. Fill in one circle only. 

1. Yes 1. Yes 

2. No 2. No 

(If No, go to #9) (If No, go to question 13) 

ScQ-09 As a general school policy, is student | ScQ-13 As a general school policy, is student 
achievement used to assign <eighth achievement used to assign eighth- 
grade> students to classes (e.g., grade students to classes (e.g., 
streaming, tracking, setting)? streaming, tracking, setting)? 

Check one circle for each line. Fill in only one circle for each row. 
1. Yes 1. Yes 
2. No 2. No 

ScQ-10 How many computers (including ScQ-14 How many computers (including 
tablets) does your school have for tablets) does your school have for 
use by <eighth grade> students? use by eighth-grade students? 

Write in the number. computers 
computers Write in the number. 

ScQ-11A Does your school have a science ScQ-15A Does your school have a science 
laboratory that can be used by laboratory that can be used by 
<eighth grade> students? eighth-grade students? 

Check one circle only. Fill in one circle only. 
1. Yes 1. Yes 
2. No 2. No 

ScQ-12 Does your school have a school ScQ-16 Does your school have a school 

library? library? 

Check one circle only. Fill in one circle only. 

1. Yes 1. Yes 

2. No 2. No 

(If No, go to #13) (If No, go to question 17) 

ScQ-15 To what degree is each of the ScQ-19 To what degree is each of the 
following a problem among <eighth following a problem among eighth- 
grade> students in your school? grade students in your school? 
Check one circle for each line. Fill in only one circle for each row. 

1. Not a problem 1. Not a problem 

2. Minor problem 2. Minor problem 

3. Moderate problem 3. Moderate problem 

4. Serious problem 4. Serious problem 

ScQ-20 In your school, are any of the 
following used to evaluate the 
practice of eighth-grade mathematics 
teachers? 
Fill in only one circle for each row. 
1. Yes 
2. No 
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Exhibit E-4. 


TIMSS 2015 Grade 8 School Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
ScQ-20a Observations by the principal or senior 
staff 
ScQ-20b Observations by inspectors or other 
persons external to the school 
ScQ-20c Student achievement 
ScQ-20d Teacher peer review 
ScQ-21 In your school, are any of the 
following used to evaluate the 
practice of eighth-grade science 
teachers? 
Fill in only one circle for each row. 
1. Yes 
2. No 
ScQ-21a Observations by the principal or senior 
staff 
ScQ-21b Observations by inspectors or other 
persons external to the school 
ScQ-21¢ Student achievement 
ScQ-21d Teacher peer review 
ScQ-16 How difficult was it to fill <eighth ScQ-22 How difficult was it to fill eighth- 
grade> teaching vacancies for this grade teaching vacancies for this 
school year for the following school year for the following 
subjects? subjects? 
Check one circle for each line. Fill in only one circle for each row. 
1. Were no vacancies in this subject 1. Were no vacancies in this subject 
2. Easy to fill vacancies 2. Easy to fill vacancies 
3. Somewhat difficult 3. Somewhat difficult 
4. Very difficult 4. Very difficult 
ScQ-17 Does your school currently use any | ScQ-23 Does your school currently use any 
incentives (e.g., pay, housing, incentives (e.g., pay, housing, 
signing bonus, smaller classes) to signing bonus, smaller classes) to 
recruit or retain <eighth grade> recruit or retain eighth-grade 
teachers in the following fields? teachers in the following fields? 
Check one circle for each line. Fill in only one circle for each row. 
1. Yes 1. Yes 
2. No 2. No 
ScQ-19 By the end of this school year, how | ScQ-25 By the end of this school year, how 
many years will you have been a many years altogether will you have 
principal altogether? been a principal? 
Please round to the nearest whole years 
number. Please round to the nearest whole 
years number. 
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Exhibit E-4. 


TIMSS 2015 Grade 8 School Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
ScQ-21 What is the highest level of formal ScQ-27 What is the highest level of formal Nat -> Int 
education you have completed? education you have completed? 1->1 
Check one circle only. Fill in one circle only. 2->2 
1. Did not complete <Bachelor's or 1. Did not complete Bachelor's degree |3->3 
equivalent level—ISCED Level 6> (4-year college program) 4->4 
2. <Bachelor's or equivalent level— 2. Bachelor's degree (4-year college 
ISCED Level 6> program) 
3. <Master's or equivalent level—ISCED 3. Master's degree or professional 
Level 7> degree (MD, DDS, lawyer, minister) 
4. <Doctor or equivalent level—ISCED 4. Doctorate (Ph.D., or Ed.D.) 
Level 8> 
ScQ-22a <Master's or equivalent level—ISCED | ScQ-28a Master's degree or professional degree 
Level 7> (MD, DDS, lawyer, minister) 
ScQ-22b <Doctor or equivalent level—ISCED ScQ-28b Doctorate (Ph.D., or Ed.D.) 
Level 8> 
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Exhibit E-5. TIMSS 2015 Grade 8 Teacher Math Questionnaire 
Questions that Require National Adaptations 
2015 International Version 2015 U.S. Adapted Version 
International National Recoding 
Item Number | Item Item Number | Item instructions 
TQG-01 What year did you start teaching? 
Please write in a year. 
TQG-01 By the end of this school year, how = |TQG-02 At the end of this school year, how 
many years will you have been many years will you have taught 
teaching altogether? altogether? 
Please round to the nearest whole years 
number. Please round to the nearest whole 
years number. 
TQG-04 What is the highest level of formal TQG-05 What is the highest level of formal Nat -> Int 
education you have completed? education you have completed? 1->1 
Check one circle only. Fill in one circle only. 2->2 
1. Did not complete <Upper secondary 1. Did not complete high school 3->4 
education—ISCED Level 3> 2. High school graduate 4->5 
2. <Upper secondary education— (If you have not completed more than 5->6 
ISCED Level 3> high school, go to question 7) 6->7 
(If you have not completed <post- 3. Associate's degree (2-year college International 
secondary or tertiary education>, program) Category 4 
go to #G6) 4. Bachelor's degree (4-year college (ISCED Level 
3. <Post-secondary, non-tertiary program) 4) is not 
education—ISCED Level 4> 5. Master's degree or professional administered 
4. <Short-cycle tertiary education— degree (MD, DDS, lawyer, minister) 
ISCED Level 5> 6. Doctorate (Ph.D., or Ed.D.) 
5. <Bachelor’s or equivalent level— 
ISCED Level 6> 
6. <Master’s or equivalent level— 
ISCED Level 7> 
7. <Doctor or equivalent level— 
ISCED Level 8> 
TQG-05 During your <post-secondary> TQG-06 During your college or university 
education, what was your major or education, what was your major or 
main area(s) of study? main area(s) of study? 
Check one circle for each line. Fill in only one circle for each row. 
1. Yes 1. Yes 
2. No 2. No 
TQG-05e <Earth Science> TQG-06e Earth Science 
TQM-13 How many <eighth grade> students /TQM-14 How many eighth-grade students 
experience difficulties understanding experience difficulties understanding 
spoken <language of test>? spoken English? 
Write in the number. students in this class 
students in this class Write in the number. 
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Exhibit E-5. 


TIMSS 2015 Grade 8 Teacher Math Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQM-16 In a typical week, how much time do |TQM-17 In a typical week, how much time do 

you spend teaching mathematics to you spend teaching mathematics to 

the students in this class? the students in this class? 

Write in the number of minutes per minutes per week 

week. Write in the number of minutes per 

Please convert the number of hours into week. 

minutes. Please convert the number of hours into 

minutes per week minutes. 

TQM-20 Which best describes the 
mathematics course you are teaching 
to the class with the TIMSS 
students? 

Fill in one circle only. 

1. Basic or general eighth-grade math 
(not algebra or pre-algebra) 

2. Pre-algebra or introduction to algebra 
3. Two-year pre-algebra 

4. Algebra | (one-year course) 

5. Algebra | (first year of a two-year 
Algebra | course) 

6. Algebra | (second year of two-year 
Algebra | course) 

7. Geometry 

8. Algebra II 

9. Integrated or sequential math 

10. Other math class 

TQM-19A Are the students in this class TQM-21A Are the students in this class 
permitted to use calculators during permitted to use calculators during 
mathematics lessons? mathematics lessons? 

Check one circle only. Fill in one circle only. 

1. Yes, with unrestricted use 1. Yes, with unrestricted use 

2. Yes, with restricted use 2. Yes, with restricted use 

3. No, calculators are not permitted 3. No, calculators are not permitted 
(If No, go to #20) (If No, go to question 22) 

TQM-20A Do the students in this class have TQM-22A Do the students in this class have 
computers (including tablets) computers (including tablets) 
available to use during their available to use during their 
mathematics lessons? mathematics lessons? 

Check one circle only. Fill in one circle only. 
1. Yes 1. Yes 
2. No 2. No 
(If No, go to #21) (If No, go to question 23) 
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Exhibit E-5. 


TIMSS 2015 Grade 8 Teacher Math Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQM-21 The following list includes the main | TQM-23 The following list includes the main 
topics addressed by the TIMSS topics addressed by the TIMSS 
mathematics test. Choose the mathematics test. Choose the 
response that best describes when response that best describes when 
the students in this class have been the students in this class have been 
taught each topic. If a topic was in taught each topic. If a topic was in 
the curriculum before the <eighth the curriculum before the eighth 
grade>, please choose “Mostly grade, please choose “Mostly taught 
taught before this year.” If a topic before this year.” If a topic was 
was taught half this year but not yet taught half this year but not yet 
completed, please choose “Mostly completed, please choose “Mostly 
taught this year.” If a topic is not in taught this year.” If a topic is not in 
the curriculum, please choose “Not the curriculum, please choose “Not 
yet taught or just introduced.” yet taught or just introduced.” 
Check one circle for each line. Fill in only one circle for each row. 
1. Mostly taught before this year 1. Mostly taught before this year 
2. Mostly taught this year 2. Mostly taught this year 
3. Not yet taught or just introduced 3. Not yet taught or just introduced 
TQM-22A How often do you usually assign TQM-24A How often do you usually assign 
mathematics homework to the mathematics homework to the 
students in this class? students in this class? 
Check one circle only. Fill in one circle only. 
1. | do not assign mathematics 1. 1 do not assign mathematics 
homework homework 
(Go to #23) (Go to question 25) 
2. Less than once a week 2. Less than once a week 
3. 1 or 2 times a week 3. 1 or 2 times a week 
4. 3 or 4 times a week 4. 3 or 4 times a week 
5. Every day 5. Every day 
TQM-23c National or regional achievement tests | TQM-25c State or district achievement tests 
TQM-25 In the past two years, how many TQM-27 In the past two years, how many 
hours in total have you spent in hours in total have you spent in 
formal <in-service/professional formal in-service/professional 
development> (e.g., workshops, development (e.g., workshops, 
seminars, etc.) for mathematics? seminars) for mathematics? 
Check one circle only. Fill in one circle only. 
1. None 1. None 
2. Less than 6 hours 2. Less than 6 hours 
3. 6-15 hours 3. 6-15 hours 
4. 16-35 hours 4. 16-35 hours 
5. More than 35 hours 5. More than 35 hours 
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Exhibit E-5. 


TIMSS 2015 Grade 8 Teacher Math Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQM-26 How well prepared do you feel you TQM-28 How well prepared do you feel you 

are to teach the following are to teach the following 

mathematics topics? mathematics topics? 

If a topic is not in the <eighth grade> If a topic is not in the eighth-grade 

curriculum or you are not curriculum or you are not 

responsible for teaching this topic, responsible for teaching this topic, 

please choose “Not applicable.” please choose “Not applicable.” 

Check one circle for each line. Fill in only one circle for each row. 

1. Not applicable 1. Not applicable 

2. Very well prepared 2. Very well prepared 

3. Somewhat prepared 3. Somewhat prepared 

4. Not well prepared 4. Not well prepared 
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Exhibit E-6. TIMSS 2015 Grade 8 Teacher Science Questionnaire 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide 


Questions that Require National Adaptations 
2015 International Version 2015 U.S. Adapted Version 
International National Recoding 
Item Number | Item Item Number | Item instructions 
TQG-01 What year did you start teaching? 
Please write in a year. 
TQG-01 By the end of this school year, how = |TQG-02 At the end of this school year, how 
many years will you have been many years will you have taught 
teaching altogether? altogether? 
Please round to the nearest whole years 
number. Please round to the nearest whole 
years number. 
TQG-04 What is the highest level of formal TQG-05 What is the highest level of formal Nat -> Int 
education you have completed? education you have completed? 1->1 
Check one circle only. Fill in one circle only. 2->2 
1. Did not complete <Upper secondary 1. Did not complete high school 3->4 
education—ISCED Level 3> 2. High school graduate 4->5 
2. <Upper secondary education— (If you have not completed more than 5->6 
ISCED Level 3> high school, go to question 7) 6->7 
(If you have not completed <post- 3. Associate's degree (2-year college International 
secondary or tertiary education>, program) Category 4 
go to #G6) 4. Bachelor's degree (4-year college (ISCED Level 
3. <Post-secondary, non-tertiary program) 4) is not 
education—ISCED Level 4> 5. Master's degree or professional administered 
4. <Short-cycle tertiary education— degree (MD, DDS, lawyer, minister) 
ISCED Level 5> 6. Doctorate (Ph.D., or Ed.D.) 
5. <Bachelor’s or equivalent level— 
ISCED Level 6> 
6. <Master’s or equivalent level— 
ISCED Level 7> 
7. <Doctor or equivalent level— 
ISCED Level 8> 
TQG-05 During your <post-secondary> TQG-06 During your college or university 
education, what was your major or education, what was your major or 
main area(s) of study? main area(s) of study? 
Check one circle for each line. Fill in only one circle for each row. 
1. Yes 1. Yes 
2. No 2. No 
TQG-05e <Earth Science> TQG-06e Earth Science 
TQS-13 How many <eighth grade> students |TQS-14 How many eighth-grade students 
experience difficulties understanding experience difficulties understanding 
spoken <language of test>? spoken English? 
Write in the number. students in this class 
students in this class Write in the number. 
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Exhibit E-6. 


TIMSS 2015 Grade 8 Teacher Science Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQS-20 Which best describes the science 

course you are teaching to the class 

with the TIMSS students? 

Fill in one circle only 

1. General science (several content 

areas of science taught separately) 

2. Integrated science (several content 

areas of science combined and taught 

together throughout the year) 

3. Life science (e.g., biology, 

ecosystems, human health) 

4. Physical science (e.g., physics or 

chemistry) 

5. Earth science (e.g., geology, Earth 

and the solar system, fossils) 

TQS-19A Do the students in this class have TQS-21A Do the students in this class have 
computers (including tablets) computers (including tablets) 
available to use during their science available to use during their science 
lessons? lessons? 

Check one circle only. Fill in one circle only. 

1. Yes 1. Yes 

2. No 2. No 

(If No, go to #20) (If No, go to question 22) 

TQS-20 The following list includes the main |TQS-22 The following list includes the main 
topics addressed by the TIMSS topics addressed by the TIMSS 
science test. Choose the response science test. Choose the response 
that best describes when the that best describes when the 
students in this class have been students in this class have been 
taught each topic. If a topic was in taught each topic. If a topic was in 
the curriculum before the <eighth the curriculum before the eighth 
grade>, please choose “Mostly grade, please choose “Mostly taught 
taught before this year.” If a topic before this year.” If a topic was 
was taught half this year but not yet taught half this year but not yet 
completed, please choose “Mostly completed, please choose “Mostly 
taught this year.” If a topic is not in taught this year.” If a topic is not in 
the curriculum, please choose “Not the curriculum, please choose “Not 
yet taught or just introduced.” yet taught or just introduced.” 
Check one circle for each line. Fill in only one circle for each row. 

1. Mostly taught before this year 1. Mostly taught before this year 
2. Mostly taught this year 2. Mostly taught this year 
3. Not yet taught or just introduced 3. Not yet taught or just introduced 
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Exhibit E-6. 


TIMSS 2015 Grade 8 Teacher Science Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQS-20Cd Electric circuits (flow of current; types of | TQS-22Cd Electric circuits (flow of current; types of 
circuits - parallel/series) and properties circuits -- parallel/series) and properties 
and uses of permanent magnets and and uses of permanent magnets and 
electromagnets electromagnets 
TQS-20Dd Earth in the solar system and the TQS-22Dd Earth in the solar system and the 
universe (phenomena on Earth - universe (phenomena on Earth -- 
day/night, tides, phases of moon, day/night, tides, phases of moon, 
eclipses, seasons; physical features of eclipses, seasons; physical features of 
Earth compared to other bodies) Earth compared to other bodies) 
TQS-21A How often do you usually assign TQS-23A How often do you usually assign 
science homework to the students in science homework to the students in 
this class? this class? 
Check one circle only. Fill in one circle only. 
1.1 do not assign science homework 1.1 do not assign science homework 
(Go to #22) (Go to question 24) 
2. Less than once a week 2. Less than once a week 
3. 1 or 2 times a week 3. 1 or 2 times a week 
4. 3 or 4 times a week 4. 3 or 4 times a week 
5. Every day 5. Every day 
TQS-22c National or regional achievement tests | TQS-24c State or district achievement tests 
TQS-24 In the past two years, how many TQS-26 In the past two years, how many 
hours in total have you spent in hours in total have you spent in 
formal <in-service/professional formal in-service/professional 
development> (e.g., workshops, development (e.g., workshops, 
seminars, etc.) for science? seminars) for science? 
Check one circle only. Fill in one circle only. 
1. None 1. None 
2. Less than 6 hours 2. Less than 6 hours 
3. 6-15 hours 3. 6-15 hours 
4. 16-35 hours 4. 16-35 hours 
5. More than 35 hours 5. More than 35 hours 
TQS-25 How well prepared do you feel you TQS-27 How well prepared do you feel you 
are to teach the following science are to teach the following science 
topics? topics? 
If a topic is not in the <eighth grade> If a topic is not in the eighth-grade 
curriculum or you are not curriculum or you are not 
responsible for teaching this topic, responsible for teaching this topic, 
please choose “Not applicable.” please choose “Not applicable.” 
Check one circle for each line. Fill in only one circle for each row. 
1. Not applicable 1. Not applicable 
2. Very well prepared 2. Very well prepared 
3. Somewhat prepared 3. Somewhat prepared 
4. Not well prepared 4. Not well prepared 
TQS-25Cd Electric circuits (flow of current; types of | TQS-27Cd Electric circuits (flow of current; types of 
circuits - parallel/series) and properties circuits -- parallel/series) and properties 
and uses of permanent magnets and and uses of permanent magnets and 
electromagnets electromagnets 
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Exhibit E-6. TIMSS 2015 Grade 8 Teacher Science Questionnaire—Continued 


Questions that Require National Adaptations 
2015 International Version 2015 U.S. Adapted Version 

International National Recoding 
Item Number | Item Item Number | Item instructions 
TQS-25Dd Earth in the solar system and the TQS-27Dd Earth in the solar system and the 

universe (phenomena on Earth - universe (phenomena on Earth -- 

day/night, tides, phases of moon, day/night, tides, phases of moon, 

eclipses, seasons; physical features of eclipses, seasons; physical features of 

Earth compared to other bodies) Earth compared to other bodies) 
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Exhibit E-7. TIMSS 2015 Grade 8 Student Questionnaire 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International 
Item Number 


Item 


National 
Item Number 


Item 


Recoding 
instructions 


SQIS-01B 


Are you Hispanic or Latino? 
Fill in one oval only. 

1. Yes, | am Hispanic or Latino 

2. No, | am not Hispanic or Latino 


SQIS-01C 


Which of the following best 
describes you? 

Fill in ovals for all that apply. 

1. White 

2. Black or African American 

3. Asian 

4. American Indian or Alaska Native 
5. Native Hawaiian or other Pacific 
Islander 


SQIS-03 


How often do you speak <language 
of test> at home? 

Fill one circle only. 

1. Always 

2. Almost always 

3. Sometimes 

4. Never 


SQIS-03a 


How often do you speak English at 
home? 

Fill in one oval only. 

1. Always 

If Always, please go to question 4. 

2. Almost always 

3. Sometimes 

4. Never 

If Almost always, Sometimes, Never, 
please go to question 3B. 


SQIS-03b 


SQIS-04 


SQIS-04a 


SQIS-04b 


SQIS-04c 


SQIS-04d 


What language do you speak at home 
(other than English)? 

Fill in one oval only. 

1. Spanish 

2. Other Please specify 

The following questions ask about 
activities you do outside of school. 
Fill in only one oval for each row 

1. Yes 

2. No 

Do you play on a sports team outside of 
school? 

Do you often play a musical instrument 
outside of school? 

Are you studying something in a class 
outside of school? 

Do you belong to a club outside of 
school (like Boy/Girl Scouts, 4-H, or 
Boys and Girls Club)? 


SQIS-05 


In this school year, are you preparing 
for or have you participated in any of 
the following activities? 

Fill in only one oval for each row. 

1. Yes 

2. No 


SQIS-05a 


Science fair 
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Exhibit E-7. TIMSS 2015 Grade 8 Student Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


Wii®, XBox®) 


Xbox) 


International National Recoding 
Item Number | Item Item Number | Item instructions 
SQIS-05b Science club 
SQIS-05c Science competition 
SQIS-06f Your own mobile phone SQIS-08f Your own cell phone 
SQIS-06g A gaming system (e.g., PlayStation®, SQIS-08g A gaming system (e.g., PlayStation, Wii, 


SQIS-06h <country-specific indicator of wealth> SQIS-08h VCR, DVD, or Blu-ray player 


5. <Short-cycle tertiary education— 
ISCED Level 5> 

6. <Bachelor’s or equivalent level— 
ISCED Level 6> 

7. <Postgraduate degree: Master’s— 
ISCED Level 7 or Doctor—ISCED Level 
8> 

8. | don’t know 


7. Doctorate (Ph.D., or Ed.D.) 
8. | don't know 


SQIS-07A What is the highest level of education | SQIS-09A What is the highest level of education | Nat -> Int 
completed by your mother (or completed by your mother (or 1->1 
stepmother or female guardian)? stepmother or female legal 2->2 
Fill one circle only. guardian)? 3->3 
1. Some <Primary education— ISCED Fill in one oval only. 4->5 
Level 1 or Lower secondary education— 1. Less than high school 5->6 
ISCED Level 2> or did not go to school 2. Some high school 6->7 
2. <Lower secondary education— 3. High school graduate 7->7 
ISCED Level 2> 4. Associate's degree (2-year college 8 ->8 
3. <Upper secondary education— program) International 
ISCED Level 3> 5. Bachelor's degree (4-year college Category 4 
4. <Post-secondary, non-tertiary program) (ISCED Level 
education—ISCED Level 4> 6. Master's degree or professional 4) is not 
5. <Short-cycle tertiary education— degree (MD, DDS, lawyer, minister) administered 
ISCED Level 5> 7. Doctorate (Ph.D., or Ed.D.) 

6. <Bachelor’s or equivalent level— 8. | don't know 
ISCED Level 6> 

7. <Postgraduate degree: Master’s— 

ISCED Level 7 or Doctor—ISCED Level 

8> 

8. | don’t know 

SQIS-07B What is the highest level of education | SQIS-09B What is the highest level of education | Nat -> Int 
completed by your father (or completed by your father (or 1->1 
stepfather or male guardian)? stepfather or male legal guardian)? |2->2 
Fill one circle only. Fill in one oval only. 3->3 
1. Some <Primary education— ISCED 1. Less than high school 4->5 
Level 1 or Lower secondary education— 2. Some high school 5->6 
ISCED Level 2> or did not go to school 3. High school graduate 6->7 
2. <Lower secondary education— 4. Associate's degree (2-year college 7->7 
ISCED Level 2> program) 8 ->8 
3. <Upper secondary education— 5. Bachelor's degree (4-year college International 
ISCED Level 3> program) Category 4 
4. <Post-secondary, non-tertiary 6. Master's degree or professional (ISCED Level 
education—ISCED Level 4> degree (MD, DDS, lawyer, minister) 4) is not 


administered 
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Exhibit E-7. TIMSS 2015 Grade 8 Student Questionnaire—Continued 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide 


Questions that Require National Adaptations 
2015 International Version 2015 U.S. Adapted Version 
International National Recoding 
Item Number | Item Item Number | Item instructions 
How far in your education do you SQIS-10 How far in your education do you Nat -> Int 
expect to go? expect to go? 1->1 
Fill one circle only. Fill in one oval only. 2->2 
1. Finish <Lower secondary education— 1. Finish middle school 3->4 
ISCED Level 2> 2. Finish high school 4->5 
2. Finish <Upper secondary education— 3. Finish Associate's degree (2-year 5 ->6 
ISCED Level 3> college program) 6->6 
3. Finish <Post-secondary, non-tertiary 4. Finish Bachelor's degree (4-year International 
education—ISCED Level 4> college program) Category 4 
4. Finish <Short-cycle tertiary 5. Finish Master's degree or (ISCED Level 
education—ISCED Level 5> professional degree (MD, DDS, lawyer, |4) is not 
5. Finish <Bachelor’s or equivalent minister) administered 
level—ISCED Level 6> 6. Finish Doctorate (Ph.D., or Ed.D.) 
6. Finish <Postgraduate degree: 
Master’s—ISCED Level 7 or Doctor— 
ISCED Level 8> 
SQIS-09A Was your mother (or stepmother or |SQIS-11A Was your mother (or stepmother or 
female guardian) born in <country>? female legal guardian) born in the 
Fill one circle only. United States? (“United States” 
1. Yes includes the 50 states, its territories, 
2. No the District of Columbia, and U.S. 
3. | don't know military bases abroad) 
Fill in one oval only. 
1. Yes 
2. No 
3. | don't know 
SQIS-09B Was your father (or stepfather or SQIS-11B Was your father (or stepfather or 
male guardian) born in <country>? male legal guardian) born in the 
Fill one circle only. United States? 
1. Yes Fill in one oval only. 
2. No 1. Yes 
3. | don't know 2. No 
3. | don't know 
SQIS-10A Were you born in <country>? SQIS-12A Were you born in the United States? 
Fill one circle only. Fill in one oval only. 
1. Yes 1. Yes 
(If Yes, go to #11) (If Yes, go to question 13) 
2. No 2. No 
SQIS-10B If No, SQIS-12B If No, 
If you were not born in <country>, If you were not born in the United 
how old were you when you came to States, how old were you when you 
<country>? came to the United States? 
Fill one circle only. Fill in one oval only. 
1. Older than 10 years old 1. Older than 10 years old 
2. 5 to 10 years old 2. 5 to 10 years old 
3. Younger than 5 years old 3. Younger than 5 years old 
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Exhibit E-7. TIMSS 2015 Grade 8 Student Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International 
Item Number 


Item 


National 
Item Number 


Item 


Recoding 
instructions 


SQIS-13B 


How many days were you absent 
from school in the last month? 
Fill in one oval only. 

1. None 

2. 1 or 2 days 

3. 3 or 4 days 

4. 5 to 10 days 

5. More than 10 days 


SQIS-14 


Have you ever repeated a grade? 
Fill in only one oval for each row. 

1. Yes 

2. No 


SQIS-14A 


In elementary school 


SQIS-14A 


In middle or junior high school 


SQIS-20c 


| need to do well in mathematics to get 
into the <university> of my choice 


SQIS-23c 


| need to do well in mathematics to get 
into the college or university of my 
choice 


SQIS-24c 


| need to do well in science to get into 
the <university> of my choice 


SQIS-27c 


| need to do well in science to get into 
the college or university of my choice 


SQIS-30 


How hard was this test compared to 
most other tests you have taken this 
year in school? 

Fill in one oval only. 

1. Easier than other tests 

2. About as hard as other tests 

3. Harder than other tests 

4. Much harder than other tests 


SQIS-31 


SQIS-32 


How hard did you try on this test 
compared to how hard you tried on 
most other tests you have taken this 
year in school? 

Fill in one oval only. 

1. Not as hard as on other tests 

2. About as hard as on other tests 

3. Harder than on other tests 

4. Much harder than on other tests 
How important was it to you to do 
well on this test? 

Fill in one oval only. 

1. Not very important 

2. Somewhat important 

3. Important 

4. Very important 
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Exhibit E-8. 


TIMSS Advanced 2015 School Questionnaire 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
ScQ-01 What is the total enrollment of ScQ-01 What is the total enrollment of 

students in your school as of <first students in your school as of March 

day of month TIMSS Advanced 1, 2015? 

testing begins, 2015>? ____ students 

Write in the number. Write in the number. 

____ Students 

ScQ-02 What is the total enrollment of ScQ-02 What is the total enrollment of 
<twelfth grade> students in your twelfth-grade students in your school 
school as of <first day of month as of March 1, 2015? 

TIMSS Advanced testing begins, students 
2015>? Write in the number. 
Write in the number. 
students 
ScQ-04 Around the 1st of October 2014, what 
percentage of students at this school 
were eligible to receive free or 
reduced-price lunches through the 
National School Lunch Program? 
percentage of students 
Write in the number. 

ScQ-04 Approximately what percentage of ScQ-05A Approximately what percentage of 
students in your school have students in your school have English 
<language of test> as their native as their native language? 
language? Fill in one circle only. 

Check one circle only. 1. More than 90% 

1. More than 90% 2. 76 to 90% 

2. 76 to 90% 3. 51 to 75% 

3. 51 to 75% 4. 26 to 50% 

4. 26 to 50% 5. 25% or less 

5. 25% or less 

ScQ-05B Of the students currently enrolled in 
your school, what percentage has 
been identified as limited-English 
proficient (LEP)/English language 
learners (ELL)? 
Fill in one circle only. 
1.0% 
2. 1-5% 
3. 6-10% 
4. 11-25% 
5. 26-50% 
6. 51-75% 
7. 76-90% 
8. Over 90% 
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Exhibit E-8. 


TIMSS Advanced 2015 School Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International 
Item Number 


Item 


National 
Item Number 


Item 


Recoding 
instructions 


ScQ-06 


What type of school is this? 

Fill in one circle only. 

1. Regular public school 

2. A regular public school with a magnet 
program 

3. A magnet school or school with a 
special program emphasis (e.g., 
Montessori, science/math school, 
performing arts school, talented/gifted 
school, foreign language immersion 
school) 

4. Special education: a school that 
primarily serves students with 
disabilities 

5. Alternative: a school designed to 
address the needs of students, typically 
at risk of educational failure, which 
cannot be met in regular schools 

6. Vocational 

7. Charter school 

8. Private (independent) 

9. Private (religiously affiliated) 

10. Other 


ScQ-06 


What percentage of <twelfth grade> 
students in your school are taking 
each of the following? 
Write in the percent. 

% 


ScQ-08 


What percentage of twelfth-grade 
students in your school are taking 
each of the following? 

% 
Write in the percent. 


ScQ-06a 


<Advanced Mathematics> 


ScQ-08A 


Advanced mathematics, such as 
calculus courses 


ScQ-06b 


<Physics> 


ScQ-08B 


Advanced physics, such as college 
preparatory physics or AP Physics 


SCQ-07 


For the <twelfth grade> students in 
your school: 


ScQ-09 


ScQ-10 


Does your school have a special 
program or track to prepare students 
for courses such as calculus or 
advanced physics? 

Fill in one circle only. 

1. Yes 

2. No 

For the twelfth-grade students in 
your school: 
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Exhibit E-8. 


TIMSS Advanced 2015 School Questionnaire—Continued 


Questions that Require National Adaptations 
2015 International Version 2015 U.S. Adapted Version 
International National Recoding 
Item Number | Item Item Number | Item instructions 
ScQ-11 To what degree is each of the ScQ-13 To what degree is each of the 
following a problem among <twelfth following a problem among twelfth- 
grade> students in your school? grade students in your school? 
Check one circle for each line. Fill in only one circle for each row. 
1. Not a problem 1. Not a problem 
2. Minor problem 2. Minor problem 
3. Moderate problem 3. Moderate problem 
4. Serious problem 4. Serious problem 
ScQ-12 How difficult was it to fill <twelfth ScQ-14 How difficult was it to fill the teaching 
grade> teaching vacancies for this vacancies for this school year for the 
school year for the following following subjects? 
subjects? Fill in only one circle for each row. 
Check one circle for each line. 1. Were no vacancies in this subject 
1. Were no vacancies in this subject 2. Easy to fill vacancies 
2. Easy to fill vacancies 3. Somewhat difficult 
3. Somewhat difficult 4. Very difficult 
4. Very difficult 
ScQ-12a Advanced mathematics ScQ-14A Advanced mathematics, such as 
calculus 
ScQ-12b Physics ScQ-14B Advanced physics, such as college 
preparatory physics or AP Physics 
ScQ-13 Does your school currently use any |ScQ-15 Does your school currently use any 
incentives (e.g., pay, housing, incentives (e.g., pay, housing, 
signing bonus, smaller classes) to signing bonus, smaller classes) to 
recruit or retain <twelfth grade> recruit or retain teachers in the 
teachers in the following fields? following fields? 
Check one circle for each line. Fill in only one circle for each row. 
1. Yes 1. Yes 
2. No 2. No 
ScQ-13a Advanced mathematics ScQ-15A Advanced mathematics, such as 
calculus 
ScQ-13b Physics ScQ-15B Advanced physics, such as college 
preparatory physics or AP Physics 
ScQ-15 By the end of this school year, how | ScQ-17 By the end of this school year, how 
many years will you have been a many years altogether will you have 
principal altogether? been a principal? 
Please round to the nearest whole years 
number. Please round to the nearest whole 
years number. 
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Exhibit E-8. 


TIMSS Advanced 2015 School Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
ScQ-17 What is the highest level of formal ScQ-19 What is the highest level of formal Nat -> Int 
education you have completed? education you have completed? 1->1 
Check one circle only. Fill in only one circle only. 2->2 
1. Did not complete <Bachelor's or 1. Did not complete Bachelor's degree |3->3 
equivalent level—ISCED Level 6> (4-year college program) 4->4 
2. <Bachelor's or equivalent level— 2. Bachelor's degree (4-year college 
ISCED Level 6> program) 
3. <Master's or equivalent level—ISCED 3. Master's degree or professional 
Level 7> degree (MD, DDS, lawyer, minister) 
4. <Doctor or equivalent level—ISCED 4. Doctorate (Ph.D., or Ed.D.) 
Level 8> 
ScQ-18a <Master's or equivalent level—ISCED | ScQ-20a Master's degree or professional degree 
Level 7> (MD, DDS, lawyer, minister) 
ScQ-18b <Doctor or equivalent level—ISCED ScQ-20b Doctorate (Ph.D., or Ed.D.) 
Level 8> 
ScQ-08A Does your school have a school Not Administered 
library? 
Check one circle only. 
1. Yes 
2. No 
(If No, go to #9) 
ScQ-08B If Yes, Not Administered 
Approximately how many books 
(print and digital) with different titles 
does your school library have 
(exclude magazines and 
periodicals)? 
Check one circle in each column. 
1. 250 or fewer 
2. 251-500 
3. 501-2,000 
4. 2,001-5,000 
5. 5,001-10,000 
6. More than 10,000 
ScQ-08Ba Print Not Administered 
ScQ-08Bb Digital Not Administered 
ScQ-08C Approximately how many titles of Not Administered 
magazines and other periodicals 
(print and digital) does your school 
library have? 
Check one circle in each column. 
1.0 
2. 1-5 
3. 6-10 
4. 11-30 
5. 31 or more 
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Exhibit E-8. TIMSS Advanced 2015 School Questionnaire—Continued 
Questions that Require National Adaptations 
2015 International Version 2015 U.S. Adapted Version 

International National Recoding 

Item Number | Item Item Number | Item instructions 
ScQ-08Ca Print Not Administered 

ScQ-08Cb Digital Not Administered 
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Exhibit E-9. TIMSS Advanced 2015 Teacher Math Questionnaire 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQG-01A What year did you start teaching? 
Please write in a year. 
TQG-01 By the end of this school year, how | TQG-01B At the end of this school year, how 
many years will you have been many years will you have taught 
teaching altogether? altogether? 
Please round to the nearest whole years 
number. Please round to the nearest whole 
years number. 
TQG-04 What is the highest level of formal TQG-04 What is the highest level of formal Nat -> Int 
education you have completed? education you have completed? 1->1 
Check one circle only. Fill in one circle only. 2->2 
1. Did not complete <tertiary> 1. Did not complete a college degree 3->3 
education (If you have not completed more than 4->4 
(If you have not completed a college degree, go to question 6) 5->5 
<tertiary> education, go to #6) 2. Associate's degree (2-year college 
2. <Short-cycle tertiary education— program) 
ISCED Level 5> 3. Bachelor's degree (4-year college 
3. <Bachelor’s or equivalent level— program) 
ISCED Level 6> 4. Master's degree or professional 
4. <Master’s or equivalent level— degree (MD, DDS, lawyer, minister) 
ISCED Level 7> 5. Doctorate (Ph.D., or Ed.D.) 
5. <Doctor or equivalent level—ISCED 
Level 8> 
TQG-05 During your <post-secondary> TQG-05 During your college or university 
education, what was your major or education, what was your major or 
main area(s) of study? main area(s) of study? 
Check one circle for each line. Fill in only one circle for each row. 
1. Yes 1. Yes 
2. No 2. No 
TQG-05e <Earth Science> TQG-05e Earth Science 
TQG-13 How many students in this class TQG-13 How many students in this class 
experience difficulties experience difficulties 
understanding spoken <language understanding spoken English? 
of test>? students in this class 
Write in the number. Write in the number. 
students in this class 
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Exhibit E-9. TIMSS Advanced 2015 Teacher Math Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQM-20A Do the students in this class have TQM-20A Do the students in this class have 

computers, tablets, calculators, or computers, tablets, calculators, or 

smartphones available to use during smartphones available to use during 

their advanced mathematics their advanced mathematics 

lessons? lessons? 

Check one circle only. Fill in one circle only. 

1. Yes 1. Yes 

2. No 2. No 

(If No, go to #21) (If No, go to question 21) 

TQM-22A Do you assign mathematics TQM-22A Do you assign mathematics 
homework to this class? homework to this class? 
Check one circle only. Fill in one circle only. 

1. Yes 1. Yes 
2. No 2. No 
(If No, go to #23) (If No, go to question 23) 

TQM-24 In the past two years, how many TQM-24 In the past two years, how many 
hours in total have you spent in hours in total have you spent in 
formal <in-service/professional formal in-service/professional 
development> (e.g., workshops, development (e.g., workshops, 
seminars, etc.) for mathematics? seminars) for mathematics? 

Check one circle only. Fill in one circle only. 
1. None 1. None 

2. Less than 6 hours 2. Less than 6 hours 
3. 6-15 hours 3. 6-15 hours 

4. 16-35 hours 4. 16-35 hours 

5. More than 35 hours 5. More than 35 hours 

TQM-26A Are you a member of <professional |TQM-26A Are you a member of the National 
organization for mathematics Council of Teachers of Mathematics 
teachers>? (NCTM) or the Mathematics 
Check one circle only. Association of America (MAA)? 

1. Yes Fill in one circle only. 
2. No 1. Yes 
2. No 

TQM-26B In the past two years, have you TQM-26B In the past two years, have you 
regularly participated in activities regularly participated in activities 
sponsored by <professional sponsored by the National Council of 
organization for mathematics Teachers of Mathematics (NCTM) or 
teachers>? the Mathematics Association of 
Check one circle only. America (MAA)? 

1. Yes Fill in one circle only. 
2. No 1. Yes 
2. No 
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Exhibit E-10._ TIMSS Advanced 2015 Teacher Physics Questionnaire 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQG-01A What year did you start teaching? 
Please write in a year. 
TQG-01 By the end of this school year, how | TQG-01B At the end of this school year, how 
many years will you have been many years will you have taught 
teaching altogether? altogether? 
Please round to the nearest whole years 
number. Please round to the nearest whole 
years number. 
TQG-04 What is the highest level of formal TQG-04 What is the highest level of formal Nat -> Int 
education you have completed? education you have completed? 1->1 
Check one circle only. Fill in one circle only. 2->2 
1. Did not complete <tertiary> 1. Did not complete a college degree 3->3 
education (If you have not completed more than 4->4 
(If you have not completed a college degree, go to question 6) 5->5 
<tertiary> education, go to #6) 2. Associate's degree (2-year college 
2. <Short-cycle tertiary education— program) 
ISCED Level 5> 3. Bachelor's degree (4-year college 
3. <Bachelor’s or equivalent level— program) 
ISCED Level 6> 4. Master's degree or professional 
4. <Master’s or equivalent level— degree (MD, DDS, lawyer, minister) 
ISCED Level 7> 5. Doctorate (Ph.D., or Ed.D.) 
5. <Doctor or equivalent level—ISCED 
Level 8> 
TQG-05 During your <post-secondary> TQG-05 During your college or university 
education, what was your major or education, what was your major or 
main area(s) of study? main area(s) of study? 
Check one circle for each line. Fill in only one circle for each row. 
1. Yes 1. Yes 
2. No 2. No 
TQG-05e <Earth Science> TQG-05e Earth Science 
TQG-13 How many students in this class TQG-13 How many students in this class 
experience difficulties experience difficulties 
understanding spoken <language understanding spoken English? 
of test>? students in this class 
Write in the number. Write in the number. 
students in this class 
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Exhibit E-10. 


TIMSS Advanced 2015 Teacher Physics Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
TQP-20A Do the students in this class have TQP-20A Do the students in this class have 

computers, tablets, calculators, or computers, tablets, calculators, or 

smartphones available to use during smartphones available to use during 

their physics lessons? their physics lessons? 

Check one circle only. Fill in one circle only. 

1. Yes 1. Yes 

2. No 2. No 

(If No, go to #21) (If No, go to question 21) 

TQP-23A Do you assign physics homework to |TQP-23A Do you assign physics homework to 

this class? this class? 

Check one circle only. Fill in one circle only. 

1. Yes 1. Yes 

2. No 2. No 

(If No, go to #24) (If No, go to question 24) 

TQP-25 In the past two years, how many TQP-25 In the past two years, how many 
hours in total have you spent in hours in total have you spent in 
formal <in-service/professional formal in-service/professional 
development> (e.g., workshops, development (e.g., workshops, 
seminars, etc.) for physics? seminars) for physics? 

Check one circle only. Fill in one circle only. 
1. None 1. None 

2. Less than 6 hours 2. Less than 6 hours 
3. 6-15 hours 3. 6-15 hours 

4. 16-35 hours 4. 16-35 hours 

5. More than 35 hours 5. More than 35 hours 

TQP-27A Are you a member of <professional |TQP-27A Are you a member of the National 
organization for physics teachers>? Science Teachers Association 
Check one circle only. (NSTA) or the American Association 
1. Yes of Physics Teachers (AAPT)? 

2. No Fill in one circle only. 
1. Yes 
2. No 

TQP-27B In the past two years, have you TQP-27B In the past two years, have you 
regularly participated in activities regularly participated in activities 
sponsored by <professional sponsored by the National Science 
organization for physics teachers>? Teachers Association (NSTA) or the 
Check one circle only. American Association of Physics 
1. Yes Teachers (AAPT)? 

2. No Fill in one circle only. 
1. Yes 
2. No 
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Exhibit E-11. 


TIMSS Advanced 2015 Student Math Questionnaire 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International 
Item Number 


Item 


National 
Item Number 


Item 


Recoding 
instructions 


SQG-01B 


Are you Hispanic or Latino? 
Fill in one oval only. 

1. Yes, | am Hispanic or Latino 

2. No, | am not Hispanic or Latino 


SQG-01C 


Which of the following best 
describes you? 

Fill in ovals for all that apply. 

1. White 

2. Black or African American 

3. Asian 

4. American Indian or Alaska Native 
5. Native Hawaiian or other Pacific 
Islander 


SQG-02 


When were you born? 
Fill the circles next to the month and 
year you were born. 


SQG-02 


When were you born? 
Fill in the ovals next to the month and 
year you were born. 


SQG-03 


How often do you speak <language 
of test> at home? 

Fill one circle only. 

1. Always 

2. Almost always 

3. Sometimes 

4. Never 


SQG-03 


How often do you speak English at 
home? 

Fill in one oval only. 

1. Always 

If Always, please go to question 4. 

2. Almost always 

3. Sometimes 

4. Never 

If Almost always, Sometimes, Never, 
please go to question 3B. 


SQG-03B 


What language do you speak at home 
(other than English)? 

Fill in one oval only. 

1. Spanish 

2. Other Please specify 


SQG-04 


How many days were you absent 
from school in the last month? 
Fill in one oval only. 

1. None 

2. 1 or 2 days 

3. 3 or 4 days 

4. 5 to 10 days 

5. More than 10 days 


SQG-05 


SQG-05a 


Have you ever repeated a grade? 
Fill in only one oval for each row. 

1. Yes 

2. No 


In elementary school 


SQG-05b 


In middle or junior high school 


SQG-05c 


In high school 
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TIMSS Advanced 2015 Student Math Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


education—ISCED Level 4> 

5. <Short-cycle tertiary education— 
ISCED Level 5> 

6. <Bachelor’s or equivalent level— 
ISCED Level 6> 

7. <Master’s or equivalent level—ISCED 
Level 7> 

8. <Doctor or equivalent level—ISCED 
Level 8> 

9. | don't know 


7. Doctorate (Ph.D., or Ed.D.) 
8. | don't know 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide 


International National Recoding 
Item Number | Item Item Number | Item instructions 
SQG-06e A gaming system (e.g., PlayStation®, SQG-08e A gaming system (e.g., PlayStation, Wii, 
Wii®, XBox®) Xbox) 
SQG-06h <country-specific indicator of wealth> SQG-08h Your own car 
SQG-07A What is the highest level of education | SQG-09A What is the highest level of education | Nat -> Int 
completed by your mother (or completed by your mother (or 1->1 
stepmother or female guardian)? stepmother or female legal 2->2 
Fill one circle only. guardian)? 3->3 
1. Some <Primary education—ISCED Fill in one oval only. 4->5 
Level 1 or 1. Less than high school 5 ->6 
Lower secondary education—ISCED 2. Some high school 6->7 
Level 2> or did not go to school 3. High school graduate 7->8 
2. <Lower secondary education— 4. Associate's degree (2-year college 8->9 
ISCED Level 2> program) International 
3. <Upper secondary education— 5. Bachelor's degree (4-year college Category 4 
ISCED Level 3> program) (ISCED Level 
4. <Post-secondary, non-tertiary 6. Master's degree or professional 4) is not 
education—ISCED Level 4> degree (MD, DDS, lawyer, minister) administered 
5. <Short-cycle tertiary education— 7. Doctorate (Ph.D., or Ed.D.) 
ISCED Level 5> 8. | don't know 
6. <Bachelor’s or equivalent level— 
ISCED Level 6> 
7. <Master’s or equivalent level—ISCED 
Level 7> 
8. <Doctor or equivalent level—ISCED 
Level 8> 
9. | don't know 
SQG-07B What is the highest level of education | SQG-09B What is the highest level of education | Nat -> Int 
completed by your father (or completed by your father (or 1->1 
stepfather or male guardian)? stepfather or male legal guardian)?  |2->2 
Fill one circle only. Fill in one oval only. 3->3 
1. Some <Primary education—ISCED 1. Less than high school 4->5 
Level 1 or 2. Some high school 5->6 
Lower secondary education—ISCED 3. High school graduate 6->7 
Level 2> or did not go to school 4. Associate's degree (2-year college 7->8 
2. <Lower secondary education— program) 8->9 
ISCED Level 2> 5. Bachelor's degree (4-year college International 
3. <Upper secondary education— program) Category 4 
ISCED Level 3> 6. Master's degree or professional (ISCED Level 
4. <Post-secondary, non-tertiary degree (MD, DDS, lawyer, minister) 4) is not 


administered 
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TIMSS Advanced 2015 Student Math Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
SQG-08 What kind of work do your father (or |SQG-10 What kind of work do your father (or 
stepfather or male guardian) and stepfather or male legal guardian) 
mother (or stepmother or female and mother (or stepmother or female 
guardian) do for their main jobs? legal guardian) do for their main 
For each, fill the circle for the job jobs? 
category that best describes what For each, fill in the oval for the job 
he/she does. Each category has a category that best describes what 
few examples to help you decide the he/she does. Each category has a 
correct category. If your father or few examples to help you decide the 
mother is not working now, think correct category. If your father or 
about the last job he/she had. mother is not working now, think 
Fill one circle in each column. about the last job he/she had. 
1. Has never worked for pay Fill in only one oval for each column. 
2. Small Business Owner 1. Has never worked for pay 
Includes owners of small businesses 2. Small Business Owner 
(fewer than 25 employees) such as Includes owners of small businesses 
retail shops, services, restaurants (fewer than 25 employees) such as 
3. Clerk retail shops, services, restaurants 
Includes office clerks; secretaries; 3. Clerk 
typists; data entry operators; customer Includes office clerks; secretaries; 
service clerks typists; data entry operators; customer 
4. Service or Sales Worker service clerks 
Includes travel attendants; restaurant 4. Service or Sales Worker 
service workers; personal care workers; Includes travel attendants; restaurant 
protective service workers; junior service workers; personal care workers; 
military and police; salespersons; street protective service workers; enlisted 
vendors military and police; salespersons; street 
5. Skilled Agricultural or Fishery Worker vendors 
Includes farmers; forestry workers; 5. Skilled Agricultural or Fishery Worker 
fishery workers; hunters and trappers Includes farmers; forestry workers; 
6. Craft or Trade Worker fishery workers; hunters and trappers 
Includes builders, carpenters, plumbers, 6. Craft or Trade Worker 
electricians, metal workers; machine Includes builders, carpenters, plumbers, 
mechanics; handicraft workers electricians, metal workers; machine 
7. Plant or Machine Operator mechanics; handicraft workers 
Includes plant and machine operators; 7. Plant or Machine Operator 
assembly-line operators; motor-vehicle Includes plant and machine operators; 
drivers assembly-line operators; motor-vehicle 
8. General Laborers drivers 
Includes domestic helpers and cleaners; 8. General Laborers 
building caretakers; messengers, Includes domestic helpers and cleaners; 
porters, and doorkeepers; farm, fishery, building caretakers; messengers, 
agricultural, and construction workers porters, and doorkeepers; farm, fishery, 
agricultural, and construction workers 
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TIMSS Advanced 2015 Student Math Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
SQG-08 9. Corporate Manager or Senior Official | SQG-10 9. Corporate Manager or Senior Official 
(Continued) Includes corporate managers such as _| (Continued) Includes corporate managers such as 
managers of large companies (25 or managers of large companies (25 or 
more employees) or managers of more employees) or managers of 
departments within large companies; departments within large companies; 
legislators or senior government legislators or senior government 
officials; senior officials of special- officials; senior officials of special- 
interest organizations; military officers interest organizations; military officers 
10. Professional 10. Professional 
Includes scientists; mathematicians; Includes scientists; mathematicians; 
computer scientists; architects; computer scientists; architects; 
engineers; life science and health engineers; life science and health 
professionals; teachers; legal professionals; teachers; legal 
professionals; social scientists; writers professionals; social scientists; writers 
and artists; religious professionals and artists; religious professionals 
11. Technician or Associate 11. Technician or Associate 
Professional Professional 
Includes science, engineering, and Includes science, engineering, and 
computer associates and technicians; computer associates and technicians; 
life science and health technicians and life science and health technicians and 
assistants; teacher aides; finance and assistants; teacher aides; finance and 
sales associate professionals; business sales associate professionals; business 
service agents; administrative assistants service agents; administrative assistants 
12. | don't know 12. | don't know 
SQG-09 How far in your education do you SQG-11 How far in your education do you Nat -> Int 
expect to go? expect to go? 1->1 
Fill one circle only. Fill in one oval only. 2->3 
1. <Upper secondary education— 1. High school 3->4 
ISCED Level 3> 2. Associate's degree (2-year college 4->5 
2. <Post-secondary, non-tertiary program) 5->6 
education—ISCED Level 4> 3. Bachelor's degree (4-year college International 
3. <Short-cycle tertiary education— program) Category 2 
ISCED Level 5> 4. Master's degree or professional (ISCED Level 
4. <Bachelor’s or equivalent level— degree (MD, DDS, lawyer, minister) 4) is not 
ISCED Level 6> 5. Doctorate (Ph.D., or Ed.D.) administered 
5. <Master’s or equivalent level—ISCED 
Level 7> 
6. <Doctor or equivalent level—ISCED 
Level 8> 
SQG-11h Actuarial Sciences SQG-13h Actuarial Sciences (i.e., uses 
mathematical and statistical methods to 
assess risk) 
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TIMSS Advanced 2015 Student Math Questionnaire—Continued 


Questions that Require National Adaptations 
2015 International Version 2015 U.S. Adapted Version 

International National Recoding 

Item Number | Item Item Number | Item instructions 

SQG-12A Was your mother (or stepmother or |SQG-14A Was your mother (or stepmother or 
female guardian) born in <country>? female legal guardian) born in the 
Fill one circle only. United States? (“United States” 

1. Yes includes the 50 states, its territories, 
2. No the District of Columbia, and U.S. 
3. | don’t know military bases abroad.) 

Fill in one oval only. 

1. Yes 

2. No 

3. | don’t know 

SQG-12B Was your father (or stepfather or SQG-14B Was your father (or stepfather or 
male guardian) born in <country>? male legal guardian) born in the 
Fill one circle only. United States? 

1. Yes Fill in one oval only. 
2. No 1. Yes 
3. | don’t know 2. No 

3. | don’t know 

SQG-13A Were you born in <country>? SQG-15A Were you born in the United States? 

Fill one circle only. Fill in one oval only. 

1. Yes 1. Yes 

2. No (If Yes, go to question 16) 
(If Yes, go to #14) 2. No 

SQG-13B If No, SQG-15B If No, 

If you were not born in <country>, If you were not born in the United 
how old were you when you came to States how old were you when you 
<country>? came to the United States? 

Fill one circle only. Fill one circle only. 

1. Older than 15 years old 1. Older than 15 years old 

2. 11 to 15 years old 2. 11 to 15 years old 

3. 5 to 10 years old 3. 5 to 10 years old 

4. Younger than 5 years old 4. Younger than 5 years old 

SQM-16A During the school year, do you work |SQM-18A During the school year, do you work 
at a paid job on a regular basis? at a paid job on a regular basis? 
Fill one circle only. Fill in one oval only. 

1. Yes 1. Yes 
2. No 2. No 
(If No, go to #17) (If No, go to question 19) 

SQM-17A During the last 12 months, have you |SQM-19A During the last 12 months, have you 
attended extra lessons or tutoring attended extra lessons or tutoring 
not provided by the school in not provided by the school in 
advanced mathematics? advanced mathematics? 

Fill one circle only. Fill in one oval only. 

1. Yes 1. Yes 

2. No 2. No 

(If No, go to #18) (If No, go to question 20) 

SQM-21e Doing well in mathematics will help me |SQM-23e Doing well in mathematics will help me 
get into the <university> of my choice get into the college or university of my 

choice 
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Questions that Require National Adaptations 


2015 International Version 2015 U.S. Adapted Version 
International National Recoding 
Item Number | Item Item Number | Item instructions 
SQM-24 How hard was this test compared to 


most other tests you have taken this 
year in school? 

Fill in one oval only. 

1. Easier than other tests 

2. About as hard as other tests 

3. Harder than other tests 

4. Much harder than other tests 
SQM-25 How hard did you try on this test 
compared to how hard you tried on 
most other tests you have taken this 
year in school? 

Fill in one oval only. 

1. Not as hard as on other tests 


2. About as hard as on other tests 
3. Harder than on other tests 
4. Much harder than on other tests 


SQM-26 How important was it to you to do 
well on this test? 

Fill in one oval only. 

1. Not very important 

2. Somewhat important 

3. Important 

4. Very important 

SQM-27 In what grade did you complete any 
of the courses listed below? 


Fill in one or more ovals in each row. 
1. Never 

2. Grade 8 or earlier 

3. Grade 9 

4. Grade 10 

5. Grade 11 

6. Grade 12 

SQM-27A Algebra | course 


SQM-27B Geometry course 
SQM-27C Algebra II course, with or without 


trigonometry 


SQM-27D Trigonometry (as a separate course) 

SQM-27E Pre-calculus course (also called 
introductory analysis) 

SQM-27F Calculus course 

SQM-27G Probability or statistics course 

SQM-27H Integrated mathematics 1 (first year of a 
multi-year course) 

SQM-271 Integrated mathematics 2 (second year 
of a multi-year course) 

SQM-27J Integrated mathematics 3 (third year of 


a multi-year course) 
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Questions that Require National Adaptations 


2015 International Version 2015 U.S. Adapted Version 
International National Recoding 
Item Number | Item Item Number | Item instructions 
SQM-27k Integrated mathematics 4 (fourth year of 
a multi-year course) 
SQM-27L Other advanced mathematics course 
SQM-28 Please indicate if you have taken or 


are currently enrolled in any of the 
following Advanced Placement (AP) 
courses. Have taken or are enrolled 


in: 

Fill in only one oval for each row. 
1. Yes 
2. No 


SQM-28A Advanced Placement (AP) Calculus AB 

SQM-28B Advanced Placement (AP) Calculus BC 
SQM-28C Advanced Placement (AP) Statistics 

SQM-29 Are you currently enrolled in or have 

you taken any online mathematics 

courses? 

Fill in one oval only. 

1. No 

2. Yes, but not for credit 

3. Yes, for high school credit 

4. Yes, for college credit 

5. Yes, for both high school and college 

credit 

SQM-30 Are you currently enrolled in or 

have you taken an International 

Baccalaureate (IB) mathematics 

course? 

Fill in one oval only. 

1. Yes 

2. No 
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Questions that Require National Adaptations 


2015 International Version 2015 U.S. Adapted Version 
International National Recoding 
Item Number | Item Item Number | Item instructions 
SQM-31 During this school year, which of the 


following have you done? 

Fill in ovals for all that apply. 

1. Taken the SAT or ACT College 
Entrance Exams 

2. Submitted the Free Application for 
Federal Student Aid (FAFSA) 

3. Applied to a 2-year college 

4. Been accepted to a 2-year college 
5. Applied to a 4-year college 

6. Been accepted to a 4-year college 
7. Talked with a military recruiter or 
contacted a ROTC program 

8. Enlisted in the military or enrolled in a 
ROTC program 

9. Applied for a full-time job 

10. Been interviewed for a full-time job 
11. None of the above 

SQM-34 During this school year, did you 
participate in any of these 
extracurricular activities? 

Fill in ovals for all that apply 

. Sports 

. Performing arts 

. Academic clubs 

. Vocational/professional clubs 
Honor societies 


. Publications 

. Student government 
. Service clubs 

. Hobby clubs 


ODNAAKR WH = 
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Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International 
Item Number 


Item 


National 
Item Number 


Item 


Recoding 
instructions 


SQG-01B 


Are you Hispanic or Latino? 
Fill in one oval only. 

1. Yes, | am Hispanic or Latino 

2. No, | am not Hispanic or Latino 


SQG-01C 


Which of the following best 
describes you? 

Fill in ovals for all that apply. 

1. White 

2. Black or African American 

3. Asian 

4. American Indian or Alaska Native 
5. Native Hawaiian or other Pacific 
Islander 


SQG-02 


When were you born? 
Fill the circles next to the month and 
year you were born. 


SQG-02 


When were you born? 
Fill in the ovals next to the month and 
year you were born. 


SQG-03 


How often do you speak <language 
of test> at home? 

Fill one circle only. 

1. Always 

2. Almost always 

3. Sometimes 

4. Never 


SQG-03 


How often do you speak English at 
home? 

Fill in one oval only. 

1. Always 

If Always, please go to question 4. 

2. Almost always 

3. Sometimes 

4. Never 

If Almost always, Sometimes, Never, 
please go to question 3B. 


SQG-03B 


What language do you speak at home 
(other than English)? 

Fill in one oval only. 

1. Spanish 

2. Other Please specify 


SQG-04 


How many days were you absent 
from school in the last month? 
Fill in one oval only. 

1. None 

2. 1 or 2 days 

3. 3 or 4 days 

4. 5 to 10 days 

5. More than 10 days 


SQG-05 


SQG-05a 


Have you ever repeated a grade? 
Fill in only one oval for each row. 

1. Yes 

2. No 


In elementary school 


SQG-05b 


In middle or junior high school 


SQG-05c 


In high school 
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TIMSS Advanced 2015 Student Physics Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


education—ISCED Level 4> 

5. <Short-cycle tertiary education— 
ISCED Level 5> 

6. <Bachelor’s or equivalent level— 
ISCED Level 6> 

7. <Master’s or equivalent level—ISCED 
Level 7> 

8. <Doctor or equivalent level—ISCED 
Level 8> 

9. | don't know 


7. Doctorate (Ph.D., or Ed.D.) 
8. | don't know 
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International National Recoding 
Item Number | Item Item Number | Item instructions 
SQG-06e A gaming system (e.g., PlayStation®, SQG-08e A gaming system (e.g., PlayStation, Wii, 
Wii®, XBox®) Xbox) 
SQG-06h <country-specific indicator of wealth> SQG-08h Your own car 
SQG-07A What is the highest level of education | SQG-09A What is the highest level of education | Nat -> Int 
completed by your mother (or completed by your mother (or 1->1 
stepmother or female guardian)? stepmother or female legal 2->2 
Fill one circle only. guardian)? 3->3 
1. Some <Primary education—ISCED Fill in one oval only. 4->5 
Level 1 or 1. Less than high school 5 ->6 
Lower secondary education—ISCED 2. Some high school 6->7 
Level 2> or did not go to school 3. High school graduate 7->8 
2. <Lower secondary education— 4. Associate's degree (2-year college 8->9 
ISCED Level 2> program) International 
3. <Upper secondary education— 5. Bachelor's degree (4-year college Category 4 
ISCED Level 3> program) (ISCED Level 
4. <Post-secondary, non-tertiary 6. Master's degree or professional 4) is not 
education—ISCED Level 4> degree (MD, DDS, lawyer, minister) administered 
5. <Short-cycle tertiary education— 7. Doctorate (Ph.D., or Ed.D.) 
ISCED Level 5> 8. | don't know 
6. <Bachelor’s or equivalent level— 
ISCED Level 6> 
7. <Master’s or equivalent level—ISCED 
Level 7> 
8. <Doctor or equivalent level—ISCED 
Level 8> 
9. | don't know 
SQG-07B What is the highest level of education | SQG-09B What is the highest level of education | Nat -> Int 
completed by your father (or completed by your father (or 1->1 
stepfather or male guardian)? stepfather or male legal guardian)?  |2->2 
Fill one circle only. Fill in one oval only. 3->3 
1. Some <Primary education—ISCED 1. Less than high school 4->5 
Level 1 or 2. Some high school 5->6 
Lower secondary education—ISCED 3. High school graduate 6->7 
Level 2> or did not go to school 4. Associate's degree (2-year college 7->8 
2. <Lower secondary education— program) 8->9 
ISCED Level 2> 5. Bachelor's degree (4-year college International 
3. <Upper secondary education— program) Category 4 
ISCED Level 3> 6. Master's degree or professional (ISCED Level 
4. <Post-secondary, non-tertiary degree (MD, DDS, lawyer, minister) 4) is not 


administered 
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2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
SQG-08 What kind of work do your father (or |SQG-10 What kind of work do your father (or 
stepfather or male guardian) and stepfather or male legal guardian) 
mother (or stepmother or female and mother (or stepmother or female 
guardian) do for their main jobs? legal guardian) do for their main 
For each, fill the circle for the job jobs? 
category that best describes what For each, fill in the oval for the job 
he/she does. Each category has a category that best describes what 
few examples to help you decide the he/she does. Each category has a 
correct category. If your father or few examples to help you decide the 
mother is not working now, think correct category. If your father or 
about the last job he/she had. mother is not working now, think 
Fill one circle in each column. about the last job he/she had. 
1. Has never worked for pay Fill in only one oval for each column. 
2. Small Business Owner 1. Has never worked for pay 
Includes owners of small businesses 2. Small Business Owner 
(fewer than 25 employees) such as Includes owners of small businesses 
retail shops, services, restaurants (fewer than 25 employees) such as 
3. Clerk retail shops, services, restaurants 
Includes office clerks; secretaries; 3. Clerk 
typists; data entry operators; customer Includes office clerks; secretaries; 
service clerks typists; data entry operators; customer 
4. Service or Sales Worker service clerks 
Includes travel attendants; restaurant 4. Service or Sales Worker 
service workers; personal care workers; Includes travel attendants; restaurant 
protective service workers; junior service workers; personal care workers; 
military and police; salespersons; street protective service workers; enlisted 
vendors military and police; salespersons; street 
5. Skilled Agricultural or Fishery Worker vendors 
Includes farmers; forestry workers; 5. Skilled Agricultural or Fishery Worker 
fishery workers; hunters and trappers Includes farmers; forestry workers; 
6. Craft or Trade Worker fishery workers; hunters and trappers 
Includes builders, carpenters, plumbers, 6. Craft or Trade Worker 
electricians, metal workers; machine Includes builders, carpenters, plumbers, 
mechanics; handicraft workers electricians, metal workers; machine 
7. Plant or Machine Operator mechanics; handicraft workers 
Includes plant and machine operators; 7. Plant or Machine Operator 
assembly-line operators; motor-vehicle Includes plant and machine operators; 
drivers assembly-line operators; motor-vehicle 
8. General Laborers drivers 
Includes domestic helpers and cleaners; 8. General Laborers 
building caretakers; messengers, Includes domestic helpers and cleaners; 
porters, and doorkeepers; farm, fishery, building caretakers; messengers, 
agricultural, and construction workers porters, and doorkeepers; farm, fishery, 
agricultural, and construction workers 
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Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
SQG-08 9. Corporate Manager or Senior Official | SQG-10 9. Corporate Manager or Senior Official 
(Continued) Includes corporate managers such as _| (Continued) Includes corporate managers such as 
managers of large companies (25 or managers of large companies (25 or 
more employees) or managers of more employees) or managers of 
departments within large companies; departments within large companies; 
legislators or senior government legislators or senior government 
officials; senior officials of special- officials; senior officials of special- 
interest organizations; military officers interest organizations; military officers 
10. Professional 10. Professional 
Includes scientists; mathematicians; Includes scientists; mathematicians; 
computer scientists; architects; computer scientists; architects; 
engineers; life science and health engineers; life science and health 
professionals; teachers; legal professionals; teachers; legal 
professionals; social scientists; writers professionals; social scientists; writers 
and artists; religious professionals and artists; religious professionals 
11. Technician or Associate 11. Technician or Associate 
Professional Professional 
Includes science, engineering, and Includes science, engineering, and 
computer associates and technicians; computer associates and technicians; 
life science and health technicians and life science and health technicians and 
assistants; teacher aides; finance and assistants; teacher aides; finance and 
sales associate professionals; business sales associate professionals; business 
service agents; administrative assistants service agents; administrative assistants 
12. | don't know 12. | don't know 
SQG-09 How far in your education do you SQG-11 How far in your education do you Nat -> Int 
expect to go? expect to go? 1->1 
Fill one circle only. Fill in one oval only. 2->3 
1. <Upper secondary education— 1. High school 3->4 
ISCED Level 3> 2. Associate's degree (2-year college 4->5 
2. <Post-secondary, non-tertiary program) 5->6 
education—ISCED Level 4> 3. Bachelor's degree (4-year college International 
3. <Short-cycle tertiary education— program) Category 2 
ISCED Level 5> 4. Master's degree or professional (ISCED Level 
4. <Bachelor’s or equivalent level— degree (MD, DDS, lawyer, minister) 4) is not 
ISCED Level 6> 5. Doctorate (Ph.D., or Ed.D.) administered 
5. <Master’s or equivalent level—ISCED 
Level 7> 
6. <Doctor or equivalent level—ISCED 
Level 8> 
SQG-11h Actuarial Sciences SQG-13h Actuarial Sciences (i.e., uses 
mathematical and statistical methods to 
assess risk) 
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TIMSS Advanced 2015 Student Physics Questionnaire—Continued 


Questions that Require National Adaptations 
2015 International Version 2015 U.S. Adapted Version 

International National Recoding 

Item Number | Item Item Number | Item instructions 

SQG-12A Was your mother (or stepmother or |SQG-14A Was your mother (or stepmother or 
female guardian) born in <country>? female legal guardian) born in the 
Fill one circle only. United States? (“United States” 

1. Yes includes the 50 states, its territories, 
2. No the District of Columbia, and U.S. 
3. | don’t know military bases abroad.) 

Fill in one oval only. 

1. Yes 

2. No 

3. | don’t know 

SQG-12B Was your father (or stepfather or SQG-14B Was your father (or stepfather or 
male guardian) born in <country>? male legal guardian) born in the 
Fill one circle only. United States? 

1. Yes Fill in one oval only. 
2. No 1. Yes 
3. | don’t know 2. No 

3. | don’t know 

SQG-13A Were you born in <country>? SQG-15A Were you born in the United States? 

Fill one circle only. Fill in one oval only. 

1. Yes 1. Yes 

2. No (If Yes, go to question 16) 
(If Yes, go to #14) 2. No 

SQG-13B If No, SQG-15B If No, 

If you were not born in <country>, If you were not born in the United 
how old were you when you came to States how old were you when you 
<country>? came to the United States? 

Fill one circle only. Fill one circle only. 

1. Older than 15 years old 1. Older than 15 years old 

2. 11 to 15 years old 2. 11 to 15 years old 

3. 5 to 10 years old 3. 5 to 10 years old 

4. Younger than 5 years old 4. Younger than 5 years old 

SQP-16A During the school year, do you work |SQP-18A During the school year, do you work 
at a paid job on a regular basis? at a paid job on a regular basis? 
Fill one circle only. Fill in one oval only. 

1. Yes 1. Yes 
2. No 2. No 
(If No, go to #17) (If No, go to question 19) 

SQP-17A During the last 12 months, have you |SQP-19A During the last 12 months, have you 
attended extra lessons or tutoring attended extra lessons or tutoring 
not provided by the school in not provided by the school in 
physics? physics? 

Fill one circle only. Fill in one oval only. 
1. Yes 1. Yes 
2. No 2. No 
(If No, go to #18) (If No, go to question 20) 
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Exhibit E-12. 


TIMSS Advanced 2015 Student Physics Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 


2015 U.S. Adapted Version 


International National Recoding 
Item Number | Item Item Number | Item instructions 
SQP-17B If Yes, SQP-19B If Yes, 
Why do you attend these extra Why did you attend these extra lessons 
lessons or tutoring? or tutoring? 
Fill one circle for each line. Fill in only one oval for each row. 
1. Yes 1. Yes 
2. No 2. No 
SQP-21e Doing well in physics will help me get SQP-23e Doing well in physics will help me get 
into the <university> of my choice into the college or university of my 
choice 
SQP-25 How hard did you try on this test 
compared to how hard you tried on 
most other tests you have taken this 
year in school? 
Fill in one oval only. 
1. Not as hard as on other tests 
2. About as hard as on other tests 
3. Harder than on other tests 
4. Much harder than on other tests 
SQP-26 How important was it to you to do 
well on this test? 
Fill in one oval only. 
1. Not very important 
2. Somewhat important 
3. Important 
4. Very important 
SQP-27 In what grade did you complete any 
of the courses listed below? 
Fill in one or more ovals in each row. 
1. Never 
2. Grade 8 or earlier 
3. Grade 9 
4. Grade 10 
5. Grade 11 
6. Grade 12 
SQP-27A General or unified science 
SQP-27B Earth and space science 
SQP-27C Life science (other than biology) 
SQP-27D Physical science (other than chemistry 
or physics) 
SQP-27E First-year biology 
SQP-27F Second-year biology 
SQP-27G First-year chemistry 
SQP-27H Second-year chemistry 
SQP-271 First-year physics 
SQP-27J Second-year physics 
SQP-27k Engineering and technology 
SQP-27L Other advanced science course 
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Exhibit E-12.  TIMSS Advanced 2015 Student Physics Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 2015 U.S. Adapted Version 
International National Recoding 
Item Number | Item Item Number | Item instructions 
SQP-28 Please indicate if you have taken or 


are currently enrolled in any of the 
following Advanced Placement (AP) 
courses. Have taken or are enrolled 
in: 
Fill in only one oval for each row. 

1. Yes 

2. No 


SQP-28A Advanced Placement (AP) Biology 


SQP-28B Advanced Placement (AP) 
Environmental Science 


SQP-28C Advanced Placement (AP) Chemistry 


SQP-28D Advanced Placement (AP) Physics B or 
Cc 


SQP-28E Advanced Placement (AP) Computer 
Science A or AB 


SQP-29 Are you currently enrolled in or have 
you taken any online science 
courses? 

Fill in one oval only 

1. No 

2. Yes, but not for credit 

3. Yes, for high school credit 

4. Yes, for college credit 

5. Yes, for both high school and college 
credit 


SQP-30 Are you currently enrolled in or 
have you taken an International 
Baccalaureate (IB) physics 
course? 

Fill in one oval only 

1. Yes 

2. No 
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Exhibit E-12.  TIMSS Advanced 2015 Student Physics Questionnaire—Continued 


Questions that Require National Adaptations 


2015 International Version 2015 U.S. Adapted Version 
International National Recoding 
Item Number | Item Item Number | Item instructions 
SQP-31 During this school year, which of the 


following have you done? 

Fill in ovals for all that apply. 

1. Taken the SAT or ACT College 
Entrance Exams 

2. Submitted the Free Application for 
Federal Student Aid (FAFSA) 

3. Applied to a 2-year college 

4. Been accepted to a 2-year college 
5. Applied to a 4-year college 

6. Been accepted to a 4-year college 
7. Talked with a military recruiter or 
contacted a ROTC program 

8. Enlisted in the military or enrolled in a 
ROTC program 

9. Applied for a full-time job 

10. Been interviewed for a full-time job 
11. None of the above 

SQP-34 During this school year, did you 
participate in any of these 
extracurricular activities? 

Fill in ovals for all that apply 

. Sports 

. Performing arts 

. Academic clubs 

. Vocational/professional clubs 
Honor societies 


. Publications 

. Student government 
. Service clubs 

. Hobby clubs 
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APPENDIX F: TIMSS 2015 NONRESPONSE 
BIAS ANALYSIS: GRADES 4 AND 8 


1. INTRODUCTION 


The Trends in International Mathematics and Science Study (TIMSS) is a large international comparative 
study of the knowledge, skills, and competencies of fourth- and eighth-grade students in the domains of 
mathematics and science. The study was carried out in more than 50 education systems, including the 
United States. The student population sampled in TIMSS is defined as all students in each country or 
other education system who were enrolled in the grade that represents four and eight years of schooling, 
counting from the first year of ISCED Level 1, provided that the mean age at the time of testing is at least 
9.5 and 13.5 years. In most participating nations, including the United States, this corresponds to all 
fourth-graders and eighth-graders. 


The U.S. TIMSS 2015 study, supported by the National Center for Education Statistics (NCES), utilized a 
two-stage stratified cluster sampling design. The first stage made use of a systematic probability- 
proportionate-to-size technique to select schools. Though efforts were made to secure the participation of 
all schools selected in the first stage, it was anticipated that not all schools would choose to participate. 
Therefore, as each school was selected in the sample, the two neighboring schools in the sorted sampling 
frame (immediately preceding and following it) were designated as replacement schools. The sampling 
frame was sorted by explicit strata and secondarily by implicit strata, so the replacement schools were 
within the same strata as the original school. If an original school refused to participate, the first 
replacement was then contacted. If that school also refused to participate, the second school was then 
contacted. 


The second stage of sampling consisted of selecting classrooms within sampled schools. At the classroom 
level, TIMSS sampled intact mathematics classes that were available to students in the target grades. 
Where feasible, two classrooms were selected per school in the United States. In schools containing only 
one class, this class was selected. The TIMSS 2015 national data collection was fielded in March, April, 
and May 2015. 


There were 300 schools in the original sample at grade 4 (hereafter referred to as TIMSS-4). Of these 
300 sampled schools, 295 were determined to be eligible! (the eligible original school sample) containing 
at least one fourth-grade class, and of these, 228 participated (the participating original sample) for an 
initial weighted response rate of 77.5 percent. An additional 22 replacement schools participated for a 
total of 250 participating schools after replacement (the participating final sample). The weighted 
response rate increased to 84.6 percent. The school participation rates for this report are summarized in 
table F-1. 


There were 300 schools in the original sample at grade 8 (hereafter referred to as TIMSS-8). Of these 

300 sampled schools, 293 were determined to be eligible? (the eligible original school sample) containing 
at least one eighth-grade class, and of these, 229 participated (the participating original sample) for an 
initial weighted response rate of 78.4 percent. An additional 17 substitute schools participated for a total 
of 246 participating schools after replacement (the participating final sample). The weighted response rate 
increased to 84.0 percent. 


' Of the 300 original schools selected for the sample, there were 5 excluded or ineligible schools at grade 4. 
? Of the 300 original schools selected for the sample, there were 7 ineligible schools at grade 8. 


U.S. TIMSS 2015 and TIMSS Advanced 2015 Technical Report and User’s Guide F-6 


The weighted student response rate for TIMSS-4 was 95.8 percent. The weighted student response rate 
for TIMSS-8 was 94.0 percent. 


Table F-1. Selected characteristics for the nonresponse bias analysis of the U.S. TIMSS grade 4 and 8 
final school samples: 2015 


Number of participating schools Percent 
Schools Eligible School participation School participation 
in schools rate before rate after 
original in Before After replacement replacement 
Grade sample sample replacement replacement Weighted Weighted 
4 300 295 228 250 77.5 84.6 
8 300 293 229 246 78.4 84.0 


SOURCE: International Association for the Evaluation of Education Achievement (IEA), Trends in International Mathematics and Science Study 
(TIMSS) 2015. 


The National Center for Education Statistics (NCES) standards for assessment surveys stipulate that a 
nonresponse bias analysis is required at any stage of data collection when the weighted unit response rate 
is less than 85 percent (before replacement). Since the U.S. TIMSS 2015 weighted school response rates 
before replacement are below 85 percent, NCES requires an investigation into the potential magnitude of 
nonresponse bias at the school level in the U.S. sample. Since the U.S. TIMSS 2015 weighted student 
response rates are above 85 percent, a nonresponse bias analysis at the student level is not required. The 
methodology used to investigate nonresponse bias in the TIMSS-4 and TIMSS-8 U.S. samples is 
provided in chapter 2 of this appendix, and the results are provided in chapters 3 and 4. 
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2. METHODOLOGY 


To measure the potential nonresponse bias at the school level, the characteristics of participating schools 
were compared to those of the total eligible sample of schools. This was conducted in a way so that the 
tests of statistical significance that were applied account for the fact that the participating schools are a 
subset of the eligible schools, and not a distinct group. 


The general approach taken involves an analysis in three parts as described below. 


a Analysis of the participating original sample: The distribution for TIMSS-4 of the 
participating original school sample (N=228) was compared with that of the total 
eligible original school sample (N=295). The distribution for TIMSS-8 of the 
participating original school sample (N=229) was compared with that of the total 
eligible original school sample (N=293).The participating original sample is the 
sample before substitution. In each sample, schools were weighted by their school 
base weights and their estimated grade 4 or 8 enrollment, excluding any nonresponse 
adjustment factor. The base weight for each original school is the reciprocal of its 
selection probability. 


7 Analysis of the participating final sample with substitutes: The distribution for 
TIMSS-4 of the participating final school sample (N=250), which includes 22 
participating substitutes that were used as replacements for nonresponding schools 
from the eligible original sample, was compared to the total eligible final school 
sample (N=295). The distribution for TIMSS-8 of the participating final school 
sample (N=246), which includes 17 participating substitutes that were used as 
replacements for nonresponding schools from the eligible original sample, was 
compared to the total eligible final school sample (N=293). The total eligible final 
sample includes the participating final sample plus those original nonrespondents that 
were not replaced by substitutes. Again, schools were weighted by their school base 
weights and their estimated grade 4 or 8 enrollment for both the eligible sample and 
the participating schools. The base weight for each substitute school is set to the base 
weight of the original school that it replaced. 


a Analysis of the nonresponse adjusted final sample with substitutes: The same sets of 
schools were compared as in the second analysis, but this time, when analyzing the 
participating final schools alone, schools were weighted by their school base weights 
and their estimated grade 4 or 8 enrollment with school nonresponse adjustments 
applied. The international weighting procedures form nonresponse adjustment classes 
by cross-classifying the explicit and implicit stratification variables. 


The first analysis indicates the potential for nonresponse bias that was introduced through school 
nonresponse. The second analysis suggests the remaining potential for nonresponse bias after the 
mitigating effects of substitution have been accounted for. The third analysis indicates the potential for 
bias after accounting for the mitigating effects of both substitution and nonresponse weight adjustments. 
Both the second and third analyses, however, may provide an overly optimistic scenario, resulting from 
the fact that substitution and nonresponse adjustments may correct somewhat for deficiencies in the 
characteristics examined here, but there is no guarantee that they are equally as effective for other 
characteristics and, in particular, for student achievement. 
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Participating TIMSS schools and the total eligible TIMSS school sample were compared on as many 
school sampling frame characteristics as possible that might provide information about the presence of 
nonresponse bias. Comparing frame characteristics between participating schools and the total eligible 
school sample is not an ideal measure of nonresponse bias if the characteristics are unrelated or weakly 
related to more substantive items in the survey; however, often it is the only approach available since 
other data are not available for nonparticipating schools. While the school-level characteristics used in 
these analyses are limited to those available in the sampling frame, each of the variables had a 
demonstrated relationship to achievement in previous TIMSS cycles. 


Frame characteristics for public schools were from the 2012-13 Common Core of Data (CCD) and, for 
private schools, from the 2011-12 Private School Universe Survey (PSS). 


The following categorical variables were available in the sampling frame for all schools: 
7 School control—indicates whether the school is under public control (operated by 


publicly elected or appointed officials) or private control (operated by privately 
elected or appointed officials and derives its major source of funds from private 


sources); 
mi Locale—urban-centric locale code (i.e., central city, suburb, town, rural); 
7 Census region—Northeast, Midwest, South and West (see Section 5. Technical Notes 


for state listing); and 


Ey Poverty level*>—for public schools, a high poverty school is defined as one in which 
50 percent or more of the students are eligible for participation in the national free and 
reduced-price lunch (FRPL) program, and a low poverty school is defined as one in 
which fewer than 50 percent are eligible. 


The following continuous variables were available in the sampling frame for all schools: 


7 Estimated number of grade 4 or grade 8 students enrolled; 
| Total number of students; and 
5 Percentage of students in seven race/ethnicity categories (White, non-Hispanic; Black, 


non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; Hawaiian/Pacific 
Islander and two or more races).* 


An additional continuous variable, the percentage of students eligible to participate in the FRPL program, 
was available only for public schools. The poverty level variable mentioned among the categorical 
variable is the recoded version of this continuous variable.° 


For categorical variables, the distribution of frame characteristics for participating schools was compared 
with the distribution for all eligible schools. The hypothesis of independence between the characteristic 
and participation status was tested using a Rao-Scott modified Chi-square statistic at the 5 percent level 


> The sample frame did not contain a direct measure of poverty. No free or reduced-price lunch (FRPL) program data were available for private 
schools, thus all private schools are treated as low-poverty schools. 

4 Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. 

> The continuous variable percentage of students eligible to participate in the FRPL program is missing for private schools; however, private 
schools are treated as low poverty for the categorical variable poverty level. 
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(Rao and Thomas 2003). For continuous variables, summary means were calculated and the difference 
between means was tested using a ¢ test. The p values for the tests are presented in the tables that follow. 
The statistical significance of differences between participants and the total eligible sample is identical to 
that which would result from comparing participants and nonparticipants, since all significance tests 
account for the fact that the participants are a subset of the full sample. The bias and relative bias are also 
shown in each table. The bias is calculated as the difference between the respective estimates for the 
participants and the eligible sample. The relative bias is calculated as the bias divided by the estimate 
from the eligible sample. The relative bias is a measure of the size of the bias compared to the eligible 
sample estimate. 


In addition to these tests, logistic regression models were used to provide a multivariate analysis that 
examined the conditional independence of these school characteristics as predictors of participation. The 
logistic regression compared frame characteristics for participating schools with non-participating schools 
which is effectively the same as comparing to the eligible same as in the bivariate analysis. It may be that 
only one or two variables are actually related to participation status. However, if these variables are also 
related to the other variables examined in the analyses, then other variables, which are not related to 
participation status, will appear as significant in simple bivariate tables. Dummy variables were created 
for each component of the categorical variables so that each component was included separately. The last 
component of each categorical variable is used as the reference category. The p value of a dummy 
variable indicates whether there is a significant difference at the 5 percent level from the effect of the 
(omitted) reference category. It is not possible to include all the frame characteristics in a single model 
because the seven race/ethnicity variables are linearly dependent (1.e., they sum up to 100 percent for 
every school). Therefore, two models were used. In the first model, six race/ethnicities (Black, non- 
Hispanic; Hispanic; Asian; American Indian or Alaska Native; Hawaiian/Pacific Islander; and two or 
more races) were included in the model with “percentage White, non-Hispanic” as the omitted category. 
In addition, an F test was used to determine whether the parameter estimates of these six characteristics 
were simultaneously equal to zero. In the second model, the summed percentage of the six race/ethnicities 
(Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; Hawaiian/Pacific Islander; 
and two or more races) replaced the six race/ethnicity variables with “percentage White, non-Hispanic” 
again as the omitted category. The second model permits the analysis of differences in the percentages of 
White, non-Hispanic students, which is not possible in the first model. All other frame characteristics 
were included in both models. 


The logistic regression was performed using WesVar® (Westat 2007) and replicate weights to properly 


account for the complex sample design. The JK2 method was used to create the replicate weights 
(Westat 2007). 
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3. RESULTS—TIMSS GRADE 4 


3.1 Original Respondent Sample 


This section presents the nonresponse bias analysis based on the original sample of 295 eligible schools 
for TIMSS-4. The distribution of the participating original sample was compared to the schools in the 
total eligible original sample. School base weights were used for both the eligible sample and the 
participating schools. The unweighted school response rate for TIMSS-4 was 77.3 percent before 
replacement, with 228 out of 295 schools participating. The weighted response rate was 77.5 percent 
before replacement. 


3.1.1 Categorical Variables (TIMSS-4) 


The distribution of participating and eligible schools by the four characteristics is shown in table F-2. The 
Chi-square statistics for school control, census region and poverty are significant and suggest that there is 
evidence of relationships with participation in the assessment. In particular, public schools were 
overrepresented among participating schools (94.3 vs. 91.3 percent, respectively), and private schools 
were underrepresented among participating schools (5.7 vs. 8.7 percent, respectively). Similarly, schools 
in the Northeast were underrepresented among participating schools relative to eligible schools (11.3 vs. 
16.5 percent, respectively), while schools in the South were overrepresented among participating schools 
(44.1 vs. 38.4 percent, respectively). Lastly, high-poverty schools were overrepresented among 
participating schools (53.2 vs. 49.3 percent, respectively), and low-poverty schools were underrepresented 
among participating schools (46.8 vs. 50.7 percent, respectively). There are no statistically significant 
relationships between participation status and locale in table F-2. 


Table F-2. Percentage distribution of eligible and participating schools in the U.S. TIMSS fourth-grade 
original sample, by selected categorical variables: 2015 


Sample schools 


Eligible Participating 
(percent) (percent) Relative Chi-square 
School characteristic (N=295) (N=228) Bias bias p value 
School control 0.006 
Public 91.3 94.3 3.05 0.033 
Private 8.7 5.7 -3.05 -0.350 
Locale 0.499 
Central city 31.8 33.3 1.47 0.046 
Suburb 39.3 37.3 -2.04 -0.052 
Town 10.9 11.4 0.52 0.048 
Rural 18.0 18.0 0.05 0.003 
Census region 0.000 
Northeast 16.5 11.3 -5.17 -0.314 
Midwest 21.3 22.2 0.88 0.041 
South 38.4 44.1 5.65 0.147 
West 23.8 22:5 -1.36 -0.057 


See notes at end of table. 
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Table F-2. Percentage distribution of eligible and participating schools in the U.S. TIMSS fourth-grade 
original sample, by selected categorical variables: 2015—Continued 


Sample schools 


Eligible Participating 
(percent) (percent) Relative Chi-square 
School characteristic (N=295) (N=228) Bias bias p value 
Poverty level 0.006 
High 49.3 53.2 3.91 0.079 
Low 50.7 46.8 -3.91 -0.077 


NOTE: Detail may not sum to totals because of rounding. Census region is the state-based region of the country (see technical notes for state 
listing). For public schools, a high poverty school is defined as one in which 50 percent or more of the students are eligible for participation in the 
FRPL program; all private schools are treated as low-poverty schools. Eligible schools contained at least one fourth-grade class. Participating 
schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools 
were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated grade 4 enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


3.1.2 Continuous Variables (TIMSS-4) 


Summary means for each continuous variable for participating and eligible schools are shown in 

tables F-3, F-4 and F-5. One participating and one nonparticipating school had missing values for 
race/ethnicity, and these schools were dropped from the analysis in table F-4. No data on FRPL eligibility 
were available for private schools, so private schools are not included in the FRPL analysis. 


There were no statistically significant differences between participating and eligible schools with respect 
to student enrollment and race/ethnicity percentage (tables F-3 and F-4). Participating schools had a 
higher mean percentage of free or reduced-price lunch students than the eligible sample (54.2 vs. 52.5 
percent, respectively; table F-5). Additionally, the absolute value of the relative bias for Hawaiian/Pacific 
Islander is greater than 10 percent, though this is due mostly to the eligible percentage being less than 1.0 
percent, as the absolute bias is small (table F-4). 


Table F-3. Mean enrollment of various characteristics for eligible and participating schools in the U.S. 
TIMSS fourth-grade original sample: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) T test 
Student enrollment (N=295) (N=228) Bias __ Relative bias p value 
Total school 564.4 560.3 -4.07 -0.007 0.696 
Fourth grade enrollment 94.9 93.0 -1.87 -0.020 0.419 


NOTE: Eligible schools contained at least one fourth-grade class. Participating schools agreed to have their students assessed. The relative bias is 
calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights that did not include a 
nonresponse adjustment factor, and by their estimated grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table F-4. Mean percentage of various characteristics for eligible and participating schools in the U.S. 
TIMSS fourth-grade original sample, by race/ethnicity: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) T test 
Race/Ethnicity (N=293) (N=227) Bias __ Relative bias p value 
White, non-Hispanic 49.9 48.7 -1.17 -0.023 0.225 
Black, non-Hispanic 14.7 15.5 0.86 0.059 0.221 
Hispanic 25-1 25.9 0.76 0.030 0.321 
Asian 4.7 4.4 -0.28 -0.060 0.311 
American Indian or Alaska 
Native 1.7 1.7 -0.07 -0.040 0.830 
Hawaiian/Pacific Islander 0.5 0.4 -0.11 -0.233 0.523 
Multiracial 3.4 3.5 0.01 0.001 0.974 


NOTE: Information on race/ethnicity is missing for two of the 295 eligible and one of the 228 participating schools in the sample. Black includes 
African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Eligible schools contained at least one fourth-grade 
class. Participating schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the 
eligible sample. Schools were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated 
grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table F-5. Mean percentage of students eligible for free or reduced-price lunch, in eligible and 
participating public schools in the U.S. TIMSS fourth-grade original sample: 2015 


Sample schools 


Eligible Participating 
(percent) (percent) t test 
Students (N=269) (N=215) Bias Relative bias p value 
Percentage of students eligible for 
free or reduced-price lunch 52:5 54.2 1.71 0.033 0.032 


NOTE: In contrast to the eighth-grade sample, information on the percentage of students eligible for free or reduced-price lunch is available for 
all the eligible public schools in the fourth-grade sample. Eligible schools contained at least one fourth-grade class. Participating schools agreed 
to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted 
by their school base weights that did not include a nonresponse adjustment factor, and by their estimated grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


3.1.3 Logistic Regression Model (TIMSS-4) 


To examine the joint relationship of various characteristics to school nonresponse, the analysis used a 
logistic regression model with participation status as the binary dependent variable and frame 
characteristics as predictor variables. Since public and private schools were modeled together using the 
variables available for all schools, the percentage of students eligible for free or reduced-price lunch was 
not included in the main logistic regression analysis. 


Standard errors and tests of hypotheses for the full model parameter estimates are presented in table F-6 
(with six race/ethnicity variables) and table F-7 (with summed race/ethnicity percentage). Private schools 
are treated as low poverty for the categorical variable poverty level. Private schools, South region and 
fourth grade enrollment were significant predictors of school participation in table F-6. The negative 
parameter estimates indicate that relative to public schools, private schools were somewhat 
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underrepresented among the participating schools, and the fourth grade enrollment in participating 
schools was smaller than in all eligible schools. The positive parameter estimate indicates that relative to 
schools in the West region, schools in the South region were somewhat overrepresented among the 
participating schools. The F test statistic to determine whether the race/ethnicity characteristics are 
simultaneously equal to 0 was 0.55 with a p value of 0.738, which indicates that no significant 
relationship with participation was detected. 


Private schools, Northeast region, and South region were significant predictors of school participation in 
table F-7. The negative parameter estimate indicates that relative to schools in the West region, schools in 
the Northeast region were somewhat underrepresented among the participating schools. 


Because the percentage of students eligible for free or reduced-price lunch was not included in the main 
logistic regression analysis, a separate analysis with public schools only was conducted. To include free 
or reduced-price lunch eligibility in a model, public schools were modeled separately using a model with 
the summed race/ethnicity percentage and adding the percentage of students eligible for free or reduced- 
price lunch. Since poverty is derived from the percentage of students eligible for free or reduced-price 
lunch, an interaction term¢ was also included in the model. Standard errors and tests of hypotheses for the 
full model parameter estimates are presented in table F-8. The Northeast region, South region, and fourth 
grade enrollment were significant predictors of school participation among public schools only. 


Table F-6. Logistic regression model parameters (with six race/ethnicity variables) using the U.S. 
TIMSS fourth-grade original school sample: 2015 


Parameter Standard t test for Ho: 
Parameter estimate error parameter = 0 p value 
Intercept 1.672 0.8428 1.9838 0.0509 
Private school -2.050 0.6395 -3.2059 0.0020 
Central city 0.435 0.7093 0.6129 0.5418 
Suburb -0.290 0.6895 -0.4203 0.6754 
Town -0.110 0.5990 -0.1834 0.8550 
Northeast -0.570 0.4450 -1.2808 0.2042 
Midwest 0.809 0.5099 1.5869 0.1167 
South 1.705 0.4763 3.5796 0.0006 
High poverty 0.347 0.5335 0.6498 0.5178 
Total school enrollment -0.001 0.0008 -0.6284 0.5316 
Fourth grade enrollment -0.005 0.0025 -2.1575 0.0342 
Black, non-Hispanic -0.013 0.0114 -1.1277 0.2630 
Hispanic 0.000 0.0093 0.0246 0.9805 
Asian -0.007 0.0186 -0.3681 0.7139 
American Indian or Alaska Native -0.017 0.0127 -1.3754 0.1731 
Hawaiian/Pacific Islander 0.000 0.2840 0.0004 0.9997 
Multiracial 0.020 0.0427 0.4662 0.6425 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated grade 4 enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


° The interaction term can be interpreted as indicating whether the marginal effect of a one percentage-point increase in FRPL is diminished or 
amplified for schools above the 50 percent cut point, relative to those below the cut point. 
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Table F-7. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS fourth-grade original school sample: 2015 


Parameter Standard t test for HO: 
Parameter estimate error parameter = 0 p value 
Intercept 1.894 0.7854 2.4112 0.0184 
Private school -1.900 0.6423 -2.9581 0.0041 
Central city 0.382 0.6974 0.5477 0.5855 
Suburb -0.363 0.6714 -0.5409 0.5902 
Town -0.089 0.5958 -0.1494 0.8816 
Northeast -0.812 0.3764 -2.1565 0.0342 
Midwest 0.511 0.5081 1.0063 0.3175 
South 1.400 0.3928 3.5635 0.0006 
High poverty 0.348 0.4957 0.7030 0.4843 
Total school enrollment 0.000 0.0007 -0.5640 0.5744 
Fourth grade enrollment -0.005 0.0026 -1.8969 0.0617 
Summed race/ethnicity percentage -0.006 0.0081 -0.6983 0.4871 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Summed race/ethnicity percentage includes Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; 
Hawaiian/Pacific Islander; and two or more races. Black includes African American, and Hispanic includes Latino. Racial categories exclude 
Hispanic origin. Schools were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated 
grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table F-8. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS fourth-grade original public school sample: 2015 


Parameter Standard t test for HO: 

Parameter estimate error parameter = 0 p value 
Intercept 1.103 1.0737 1.0274 0.3075 
Central city 0.315 0.7152 0.4401 0.6612 
Suburb -0.492 0.6745 -0.7294 0.4681 
Town 0.196 0.6648 0.2953 0.7686 
Northeast -0.869 0.4078 -2.1321 0.0363 
Midwest 0.312 0.4935 0.6324 0.5290 
South 1.378 0.4557 3.0249 0.0034 
High poverty 1.474 1.6272 0.9056 0.3680 
Free or reduced-price lunch eligibility 0.027 0.0187 1.4198 0.1598 
High poverty * free or reduced-price 

lunch eligibility -0.032 0.0298 -1.0595 0.2928 
Total school enrollment 0.000 0.0011 0.1539 0.8781 
Fourth grade enrollment -0.006 0.0026 -2.4737 0.0156 
Summed race/ethnicity percentage -0.008 0.0088 -0.8904 0.3761 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program. Black includes African American, 
and Hispanic includes Latino. Summed race/ethnicity percentage includes Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska 
Native; Hawaiian/Pacific Islander; and two or more races. Racial categories exclude Hispanic origin. Schools were weighted by their school base 
weights that did not include a nonresponse adjustment factor, and by their estimated grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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3.2 Participating Final Sample with Substitutes 


This section presents the nonresponse bias analysis based on the final sample of 295 eligible schools for 
TIMSS-4 including participating substitute schools. The distribution of the participating final sample was 
compared to the schools in the total eligible final sample. The total eligible final sample includes 
participating final sample plus those original nonrespondents who were not replaced by substitutes. 
School base weights were used for both the eligible sample and the participating schools. Through the use 
of substitute schools, the unweighted school response rate for TIMSS-4 was 84.7 percent after 
replacement, with 250 out of 295 schools participating. The weighted response rate was 84.6 percent after 
replacement. 


3.2.1 Categorical Variables (TIMSS-4) 


The distribution of participating and eligible schools by the four characteristics is shown in table F-9. The 
Chi-square statistic for census region is significant and suggests that there is evidence of a relationship 
with participation in the assessment. In particular, schools in the Northeast were underrepresented among 
participating schools relative to eligible schools (12.3 vs. 16.5 percent, respectively), while schools in the 
South were overrepresented among participating schools (43.0 vs. 38.4 percent, respectively). There are 
no statistically significant relationships between participation status and any of the other characteristics 
shown in table F-9. Additionally, the absolute value of the relative bias for private schools is greater than 
10 percent, which indicates potential bias for school control though substantially reduced after 
substitution (table F-9). Note that the relative bias for private schools is much higher than for public 
schools due to the binary nature of the variable as the absolute bias is the same for both public and 
private. 


Table F-9. Percentage distribution of eligible and participating schools in the U.S. TIMSS fourth-grade 
final sample, by selected categorical variables: 2015 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=295) (N=250) Bias bias p value 

School control 0.207 
Public 91.3 92.5 1.18 0.013 
Private 8.7 75 -1.18 -0.136 

Locale 0.942 
Central city 31.8 32.0 0.14 0.004 
Suburb 39.3 38.8 -0.51 -0.013 
Town 10.9 10.8 -0.08 -0.007 
Rural 18.0 18.4 0.45 0.025 

Census region 0.000 
Northeast 16.5 12.3 -4.17 -0.254 
Midwest 21.3 23.1 1.78 0.083 
South 38.4 43.0 4.54 0.118 
West 23.8 21.7 -2.14 -0.090 


See notes at end of table. 
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Table F-9. Percentage distribution of eligible and participating schools in the U.S. TIMSS fourth-grade 
final sample, by selected categorical variables: 2015—Continued 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=295) (N=250) Bias bias p value 

Poverty level 0.162 
High 49.3 50.9 1.65 0.033 
Low 50.7 49.1 -1.65 -0.033 


NOTE: Detail may not sum to totals because of rounding. Census region is the state-based region of the country (see technical notes for state 
listing). For public schools, a high poverty school is defined as one in which 50 percent or more of the students are eligible for participation in the 
FRPL program; all private schools are treated as low poverty schools. Eligible schools contained at least one fourth-grade class. Participating 
schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools 
were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated grade 4 enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


3.2.2 Continuous Variables (TIMSS-4) 


Summary means for each continuous variable for participating and eligible schools are shown in 

tables F-10, F-11, and F-12. One participating and one nonparticipating school had missing values for 
race/ethnicity, and these schools were dropped from the analysis in table F-11. No data on FRPL 
eligibility was available for private schools, and so private schools are not included in the FRPL analysis. 


There were no statistically significant differences between participating and eligible schools with respect 
to student enrollment, race/ethnicity percentage or free or reduced-price lunch (tables F-10, F-11, and 
F-12). However, the absolute value of the relative bias for American Indian or Alaska Native is greater 
than 10 percent, though this is due mostly to the eligible percentage being less than 2.0 percent, as the 
absolute bias is small (table F-11). 


Table F-10. Mean enrollment of various characteristics for eligible and participating schools in the U.S. 
TIMSS fourth-grade final sample: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) t test 
Student enrollment (N=295) (N=250) Bias __ Relative bias p value 
Total school 559.6 565.5 5.86 0.010 0.325 
Fourth grade enrollment 94.8 93.9 -0.88 -0.009 0.663 


NOTE: Eligible schools contained at least one fourth-grade class. Participating schools agreed to have their students assessed. The relative bias is 
calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights that did not include a 
nonresponse adjustment factor, and by their estimated grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table F-11. Mean percentage of various characteristics for eligible and participating schools in the U.S. 
TIMSS fourth-grade final sample, by race/ethnicity: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) t test 
Race/ethnicity (N=293) (N=249) Bias _ Relative bias p value 
White, non-Hispanic 49.8 49.5 -0.26 -0.005 0.726 
Black, non-Hispanic 14.2 14.8 0.62 0.044 0.237 
Hispanic 25.4 25.4 0.01 0.000 0.984 
Asian 5.0 4.8 -0.23 -0.046 0.356 
American Indian or Alaska 
Native 1.7 1.5 -0.19 -0.109 0.554 
Hawaiian/Pacific Islander 0.4 0.4 0.03 0.074 0.258 
Multiracial 3.4 3.4 0.02 0.006 0.855 


NOTE: Information on race/ethnicity is missing for two of the 295 eligible and one of the 250 participating schools in the sample. Black includes 
African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Eligible schools contained at least one fourth-grade 
class. Participating schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the 
eligible sample. Schools were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated 
grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table F-12. Mean percentage of students eligible for free or reduced-price lunch, in eligible and 
participating public schools in the U.S. TIMSS fourth-grade final sample: 2015 


Sample schools 


Eligible Participating 
(percent) (percent) t test 
Students (N=269) (N=231) Bias _ Relative bias p value 


Percentage of students 
eligible for free or 
reduced-price lunch 52.5 53.1 0.59 0.011 0.402 


NOTE: In contrast to the eighth-grade sample, information on the percentage of students eligible for free or reduced-price lunch is available for 
all the eligible public schools in the fourth-grade sample. Eligible schools contained at least one fourth-grade class. Participating schools agreed 
to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted 
by their school base weights that did not include a nonresponse adjustment factor, and by their estimated grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


3.2.3 Logistic Regression Model (TIMSS-4) 


To examine the joint relationship of various characteristics to school nonresponse, the analysis used a 
logistic regression model with participation status as the binary dependent variable and frame 
characteristics as predictor variables. Since public and private schools were modeled together using the 
variables available for all schools, the percentage of students eligible for free or reduced-price lunch was 
not included in the main logistic regression analysis. 


Standard errors and tests of hypotheses for the full model parameter estimates are presented in table F-13 
(with six race/ethnicity variables) and table F-14 (with summed race/ethnicity percentage). Private 
schools are treated as low poverty for the categorical variable poverty level. The Midwest region, South 
region, fourth grade enrollment and the percentage of American Indian or Alaska Native students were 
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significant predictors of school participation in table F-13. The positive parameters estimate indicates that 
relative to schools in the West region, schools in the Midwest and South regions were somewhat 
overrepresented among the participating schools. The negative parameter estimates indicate that the 
fourth grade enrollment and the percentage of American Indian or Alaska Native students in participating 
schools were smaller than in all eligible schools. The F test statistic to determine whether the 
race/ethnicity characteristics are simultaneously equal to 0 was 1.50 with a p value of 0.200, which 
indicates no significant relationship was detected with participation. 


The Midwest region, South region, and fourth grade enrollment were again significant predictors of 
school participation in table F-14. 


Because the percentage of students eligible for free or reduced-price lunch was not included in the main 
logistic regression analysis, a separate analysis with public schools only was conducted. To include free 
or reduced-price lunch eligibility in a model, public schools were modeled separately using a model with 
the summed race/ethnicity percentage and adding the percentage of students eligible for free or reduced- 
price lunch. Since poverty is derived from the percentage of students eligible for free or reduced-price 
lunch, an interaction term was also included in the model. Standard errors and tests of hypotheses for the 
full model parameter estimates are presented in table F-15. The South region and fourth grade enrollment 
were significant predictors of school participation. The model with the six race/ethnicity variables is not 
shown due to complex interactions that make the results difficult to interpret. 


Table F-13. Logistic regression model parameters (with six race/ethnicity variables) using the U.S. 
TIMSS fourth-grade final school sample: 2015 


Parameter Standard t test for Ho: 
Parameter estimate error parameter = 0 p value 
Intercept 0.526 0.8932 0.5894 0.5574 
Private school -1.192 0.8078 -1.4757 0.1442 
Central city 0.217 0.7992 0.2715 0.7868 
Suburb 0.431 0.8067 0.5348 0.5943 
Town 0.275 0.6727 0.4087 0.6840 
Northeast -0.136 0.4834 -0.2819 0.7788 
Midwest 1.844 0.6126 3.0106 0.0036 
South 2.551 0.6490 3.9310 0.0002 
High poverty 0.168 0.5551 0.3028 0.7629 
Total school enrollment 0.001 0.0010 1.3179 0.1916 
Fourth grade enrollment -0.007 0.0035 -2.1165 0.0376 
Black, non-Hispanic -0.010 0.0133 -0.7732 0.4418 
Hispanic 0.003 0.0105 0.2475 0.8052 
Asian -0.010 0.0243 -0.4021 0.6888 
American Indian or Alaska Native -0.022 0.0101 -2.2175 0.0296 
Hawaiian/Pacific Islander 0.439 0.3685 1.1925 0.2368 
Multiracial 0.028 0.0494 0.5773 0.5655 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated grade 4 enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide F-19 


Table F-14. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS fourth-grade final school sample: 2015 


Parameter Standard t test for Ho: 
Parameter estimate error parameter = 0 p value 
Intercept 0.916 0.807 1.1346 0.2601 
Private school -1.027 0.802 -1.2809 0.2042 
Central city 0.216 0.7674 0.2811 0.7794 
Suburb 0.317 0.7831 0.405 0.6866 
Town 0.302 0.6684 0.4513 0.6531 
Northeast -0.598 0.419 -1.4272 0.1577 
Midwest 1.372 0.6348 2.1605 0.0339 
South 2.037 0.5799 3.5123 0.0008 
High poverty 0.057 0.5133 0.1104 0.9124 
Total school enrollment 0.002 0.0009 1.7508 0.0841 
Fourth grade enrollment -0.007 0.0034 -2.1319 0.0363 
Summed race/ethnicity percentage -0.002 0.0082 -0.2509 0.8026 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Summed race/ethnicity percentage includes Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; 
Hawaiian/Pacific Islander; and two or more races. Black includes African American, and Hispanic includes Latino. Racial categories exclude 
Hispanic origin. Schools were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated 
grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table F-15. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS fourth-grade final public school sample: 2015 


Parameter Standard t test for HO: 

Parameter estimate error parameter = 0 p value 
Intercept 0.750 1.0463 0.7166 0.4759 
Central city 0.269 0.8104 0.3317 0.7410 
Suburb 0.000 0.7465 0.0001 0.9999 
Town 0.540 0.7451 0.7253 0.4705 
Northeast -0.621 0.4306 -1.4428 0.1532 
Midwest 1.310 0.6710 1.9527 0.0546 
South 2.146 0.6237 3.4414 0.0009 
High poverty 1.570 2.0725 0.7577 0.4510 
Free or reduced-price lunch eligibility 0.015 0.0195 0.7907 0.4316 
High poverty * free or reduced-price lunch 

eligibility -0.028 0.0333 -0.8382 0.4046 
Total school enrollment 0.001 0.0012 1.1061 0.2722 
Fourth grade enrollment -0.007 0.0037 -1.9935 0.0498 
Summed race/ethnicity percentage -0.006 0.0096 -0.6266 0.5329 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program. Black includes African American, 
and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were weighted by their school base weights that did not include 
a nonresponse adjustment factor, and by their estimated grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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33 Nonresponse-adjusted Final Sample with Substitutes 


This section presents the nonresponse bias analysis based on the final sample of 295 eligible schools for 
TIMSS-4. The distribution of the participating final sample, including participating substitute schools, 
was compared to the schools in the total eligible final sample, just like the previous section. However, in 
the analyses that follow, school base weights were used for the eligible sample of schools, whereas 
nonresponse-adjusted weights were used for the participating schools. 


3.3.1 Categorical Variables (TIMSS-4) 


The distribution of participating and eligible schools by the four characteristics is shown in table F-16. 
There are no statistically significant relationships between participation status and any of the other 
characteristics shown in table F-16. 


Table F-16. Percentage distribution of eligible and participating schools in the U.S. TIMSS fourth-grade 
nonresponse-adjusted sample, by selected categorical variables: 2015 


Sample schools 


Eligible Participating 
(percent) (percent) Relative Chi-square 
School characteristic (N=295) (N=250) Bias bias p value 
School control 1.000 
Public 91.3 91.3 0.00 0.000 
Private 8.7 8.7 0.00 0.000 
Locale 0.831 
Central city 31.8 32.7 0.84 0.026 
Suburb 39.3 39.0 -0.36 -0.009 
Town 10.9 10.3 -0.56 -0.051 
Rural 18.0 18.1 0.08 0.004 
Census region 1.000 
Northeast 16.5 16.5 0.00 0.000 
Midwest 21.3 21.3 0.00 0.000 
South 38.4 38.4 0.00 0.000 
West 23.8 23.8 0.00 0.000 
Poverty level 1.000 
High 49.3 49.3 0.00 0.000 
Low 50.7 50.7 0.00 0.000 


NOTE: Detail may not sum to totals because of rounding. Census region is the state-based region of the country (see technical notes for state 
listing). For public schools, a high poverty school is defined as one in which 50 percent or more of the students are eligible for participation in the 
FRPL program; all private schools are treated as low poverty schools. Eligible schools contained at least one fourth-grade class. Participating 
schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools 
were weighted by their school nonresponse adjusted weight and by their estimated grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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3.3.2 Continuous Variables (TIMSS-4) 


Summary means for each continuous variable for participating and eligible schools are shown in 

tables F-17, F-18 and F-19. One participating and one nonparticipating school had missing values for 
race/ethnicity, and these schools were dropped from the analysis in table F-18. No data on FRPL 
eligibility was available for private schools, and so private schools are not included in the FRPL analysis. 


There were no statistically significant differences between participating and eligible schools with respect 
to student enrollment, race/ethnicity percentage or free or reduced-price lunch (tables F-17, F-18, and 
F-19). However, the absolute values of the relative bias for Hawaiian/Pacific Islander and American 
Indian or Alaska Native are greater than 10 percent, though this is due mostly to the eligible percentages 
being less than 2.0 percent, as the absolute biases are small (table F-18). 


Table F-17._ Mean enrollment of various characteristics for eligible and participating schools in the U.S. 
TIMSS fourth-grade nonresponse-adjusted sample: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) t test 
Student enrollment (N=295) (N=250) Bias __ Relative bias p value 
Total school 559.6 563.2 3.55 0.006 0.614 
Fourth grade enrollment 94.8 92.7 -2.15 -0.023 0.301 


NOTE: Eligible schools contained at least one fourth-grade class. Participating schools agreed to have their students assessed. The relative bias is 
calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school nonresponse adjusted weight and 
by their estimated grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table F-18. Mean percentage of various characteristics for eligible and participating schools in the U.S. 
TIMSS fourth-grade nonresponse-adjusted sample, by race/ethnicity: 2015 


Sample schools 


Eligible Participating 

(mean) (mean) t test 
Race/ethnicity (N=293) (N=249) Bias __ Relative bias p value 
White, non-Hispanic 49.8 49.9 0.07 0.001 0.930 
Black, non-Hispanic 14.2 14.2 -0.05 -0.003 0.931 
Hispanic 25.4 25.5 0.05 0.002 0.944 
Asian 5.0 5.1 0.04 0.007 0.906 

American Indian or Alaska 
Native 1.7 1.5 -0.27 -0.157 0.394 
Hawaiian/Pacific Islander 0.4 0.5 0.07 0.185 0.124 


Multiracial 3.4 3.5 0.09 0.026 0.642 


NOTE: Information on race/ethnicity is missing for two of the 295 eligible and one of the 250 participating schools in the sample. Black includes 
African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Eligible schools contained at least one fourth-grade 
class. Participating schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the 
eligible sample. Schools were weighted by their school nonresponse adjusted weight and by their estimated grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table F-19. Mean percentage of students eligible for free or reduced-price lunch, in eligible and 
participating public schools in the U.S. TIMSS fourth-grade nonresponse-adjusted sample: 
2015 


Sample schools 
Eligible Participating 
(percent) (percent) t test 
Students (N=269) (N=231) Bias _ Relative bias p value 
Percentage of students 
eligible for free or 
reduced-price lunch 52.5 52.6 0.06 0.001 0.942 


NOTE: In contrast to the eighth-grade sample, information on the percentage of students eligible for free or reduced-price lunch is available for 
all the eligible public schools in the fourth-grade sample. Eligible schools contained at least one fourth-grade class. Participating schools agreed 
to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted 
by their school nonresponse adjusted weights and by their estimated grade 4 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


3.4 Summary—Grade 4 


The investigation into nonresponse bias at the school level for the U.S. TIMSS-4 effort shows statistically 
significant relationships between response status and some of the available school characteristics that 
were examined in the analysis. 


For original sample schools, four variables were found to be statistically significantly related to 
participation in the bivariate analysis: school control (table F-2); Census region (table F-2); poverty (table 
F-2) and free or reduced-price lunch (table F-5). Additionally, the absolute value of the relative bias for 
Hawaiian/Pacific Islander is greater than 10 percent, though this is due mostly to the eligible percentage 
being less than 2.0 percent, as the absolute bias is small (table F-4). Although each of these findings 
indicates some potential for nonresponse bias, when all of these factors were considered simultaneously 
in a regression analysis, private schools, South region and fourth grade enrollment were significant 
predictors of participation (table F-6). The second model showed that private schools, Northeast region, 
and South region were significant predictors of participation (table F-7, with summed race/ethnicity 
percentage). The third model showed Northeast, South regions and fourth grade enrollment were 
significant predictors of school participation among public schools only (table F-8). 


For final sample schools (with substitutes), only Census region (table F-9) was found to be statistically 
significantly related to participation in the bivariate analysis. Additionally, the absolute value of the 
relative bias for private schools is greater than 10 percent, which indicates potential bias for school 
control though substantially reduced after substitution (table F-9). When all of these factors were 
considered simultaneously in a regression analysis, Midwest region, South region, fourth grade 
enrollment and the percentage of American Indian or Alaska Native students were significant predictors 
of participation (table F-13). The second model showed that Midwest region, South region, and fourth 
grade enrollment were significant predictors of participation (table F-14, with summed race/ethnicity 
percentage). The third model showed South region and fourth grade enrollment were a significant 
predictors of school participation among public schools only (table F-15). 


For final sample schools with school nonresponse adjustments applied to the weights, no variables were 


found to be statistically significantly related to participation in the bivariate analysis. However, the 
absolute values of the relative bias for Hawaiian/Pacific Islander and American Indian or Alaska Native 
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are greater than 10 percent , though this is due mostly to the eligible percentages being less than 2.0 
percent, as the absolute biases are small (table F-18). The multivariate regression analysis cannot be 
conducted after the school nonresponse adjustments are applied to the weights. The concept of 
nonresponse-adjusted weights does not apply to the nonresponding units, and, thus, we cannot conduct an 
analysis that compares respondents with nonrespondents using nonresponse-adjusted weights. 


The results of the analyses are summarized in table F-20. 


Table F-20. Characteristics with p values less than 0.05 and absolute relative bias greater than 10 
percent, U.S. TIMSS fourth-grade schools: 2015 


Analysis 
Original sample 


Regression model a 


Regression model b 


Regression model c 


Sample with substitutes 


Regression model a 


Regression model b 


Regression model c 


Nonresponse adjusted 


Characteristics with p values 
less than 0.05 


School control, Census region, poverty, 
free or reduced-price lunch 


Private schools, South region, fourth 
grade enrollment 


Private schools, Northeast region, South 
region 


Northeast region, South region, fourth 
grade enrollment 


Census region 
South region, Midwest region, fourth 
grade enrollment, American Indian or 


Alaska Native 


South region, Midwest region, fourth 
grade enrollment 


South region, fourth grade enrollment 


None 


Additional characteristics with absolute 
relative bias greater than 10 percent 


Hawaiian/Pacific Islander 


N/a 


Private schools 


N/a 


N/a 


N/a 


Hawaiian/Pacific Islander, American 
Indian or Alaska Native 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS) 2015. 


These results suggest that there is some potential for nonresponse bias in the U.S. TIMSS-4 original 
sample based on the characteristics studied. It also suggests that, while there is some evidence that the use 
of substitute schools reduced the potential for bias, it has not reduced it substantially. However, after the 
application of school nonresponse adjustments, there is no evidence of resulting potential bias in the 
available frame variables and correlated variables in the final sample. 
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4. RESULTS—TIMSS GRADE 8 


4.1 Original Respondent Sample 


This section presents the nonresponse bias analysis based on the original sample of 293 eligible schools 
for TIMSS-8. The distribution of the participating original sample was compared to the schools in the 
total eligible original sample. School base weights were used for both the eligible sample and the 
participating schools. The unweighted school response rate for TIMSS-8 was 78.2 percent before 
replacement, with 229 out of 293 schools participating. The weighted response rate was 78.4 percent 
before replacement. 


4.1.1 Categorical Variables (TIMSS-8) 


The distribution of participating and eligible schools by the four characteristics is shown in table F-21. 
The Chi-square statistics for Census region and poverty are significant and suggest that there is evidence 
of relationships with participation in the assessment. In particular, schools in the Northeast were 
underrepresented among participating schools relative to eligible schools (13.9 vs. 17.0 percent, 
respectively), while schools in the South were overrepresented among participating schools (42.1 vs. 38.4 
percent, respectively). High-poverty schools were overrepresented among participating schools (48.9 vs. 
45.1 percent, respectively), and low-poverty schools were underrepresented among participating schools 
(51.1 vs. 54.9 percent, respectively). There are no statistically significant relationships between 
participation status and any of the other characteristics shown in table F-21. Additionally, the absolute 
value of the relative bias for private schools is greater than 10 percent, which indicates potential bias for 
school control, though the absolute bias is relatively small. Note that the relative bias for private schools 
is much higher than for public schools due to the binary nature of the variable as the absolute bias is the 
same for both public and private. 


Table F-21. Percentage distribution of eligible and participating schools in the U.S. TIMSS eighth-grade 
original sample, by selected categorical variables: 2015 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=293) (N=229) Bias bias p value 

School control 0.368 
Public 92.0 92.9 0.84 0.009 
Private 8.0 71 -0.84 -0.105 

Locale 0.211 
Central city 30.2 32.0 1.88 0.062 
Suburb 39.9 36.7 -3.21 -0.080 
Town 11.3 11.4 0.09 0.008 
Rural 18.6 19.8 1.23 0.066 


See notes at end of table. 
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Table F-21. Percentage distribution of eligible and participating schools in the U.S. TIMSS eighth-grade 
original sample, by selected categorical variables: 2015—Continued 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=293) (N=229) Bias bias p value 

Census region 0.015 
Northeast 17.0 13.9 -3.05 -0.180 
Midwest 21.6 22.0 0.37 0.017 
South 38.4 42.1 3.70 0.096 
West 23.0 22.0 -1.02 -0.044 

Poverty level 0.013 
High 45.1 48.9 3.81 0.085 
Low 54.9 51.1 -3.81 -0.069 


NOTE: Detail may not sum to totals because of rounding. Census region is the state-based region of the country (see technical notes for state 
listing). For public schools, a high poverty school is defined as one in which 50 percent or more of the students are eligible for participation in the 
FRPL program; all private schools are treated as low-poverty schools Eligible schools contained at least one eighth-grade class. Participating 
schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools 
were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated grade 8 enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


4.1.2 Continuous Variables (TIMSS-8) 


Summary means for each continuous variable for participating and eligible schools are shown in 

tables F-22, F-23 and F-24. One participating school had missing values for race/ethnicity, and this school 
was dropped from the analysis in table F-23. No data on FRPL eligibility was available for private 
schools, and so private schools are not included in the FRPL analysis. Three participating schools had 
missing values for free or reduced-price lunch, and these schools were dropped from the analysis in table 
F-24. 


There were no statistically significant differences between participating and eligible schools with respect 
to student enrollment, race/ethnicity percentage or free or reduced-price lunch (tables F-22, F-23 and 
F-24). 


Table F-22. Mean enrollment of various characteristics for eligible and participating schools in the U.S. 
TIMSS eighth-grade original sample: 2015 


Sample schools 


Eligible Participating 

(mean) (mean) t test 
Student enrollment (N=293) (N=229) Bias __ Relative bias p value 
Total school 745.5 757.6 12.02 0.016 0.309 
Eighth grade enrollment 242.4 244.5 2.18 0.009 0.659 


NOTE: Eligible schools contained at least one eighth-grade class. Participating schools agreed to have their students assessed. The relative bias is 
calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights that did not include a 
nonresponse adjustment factor, and by their estimated grade 8 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table F-23. Mean percentage of various characteristics for eligible and participating schools in the U.S. 
TIMSS eighth-grade original sample, by race/ethnicity: 2015 


Sample schools 


Eligible Participating 

(mean) (mean) t test 
Race/ethnicity (N=292) (N=228) Bias _ Relative bias p value 
White, non-Hispanic 52.9 52.3 -0.55 -0.010 0.559 
Black, non-Hispanic 14.9 15.5 0.57 0.039 0.400 
Hispanic 23.9 24.1 0.23 0.010 0.754 
Asian 4.8 4.5 -0.22 -0.046 0.301 

American Indian or Alaska 
Native 0.7 0.7 -0.01 -0.012 0.886 
Hawaiian/Pacific Islander 0.3 0.3 0.02 0.082 0.461 
Multiracial 2.6 2.5 -0.05 -0.020 0.628 


NOTE: Information on race/ethnicity is missing for one of the 229 participating schools in the sample. Black includes African American, and 
Hispanic includes Latino. Racial categories exclude Hispanic origin. Eligible schools contained at least one eighth-grade class. Participating 
schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools 
were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated grade 8 enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table F-24. Mean percentage of students eligible for free or reduced-price lunch, in eligible and 
participating public schools in the U.S. TIMSS eighth-grade original sample: 2015 


Sample schools 


Eligible _— Participating 


(percent) (percent) t test 

Students (N=267) (N=210) Bias _ Relative bias p value 
Percentage of students eligible for 

free or reduced-price lunch 51.2 52.7 1.46 0.029 0.095 


NOTE: Information on percentage of students eligible for free or reduced-price lunch is missing for three of the 213 participating public schools 
in the sample. Eligible schools contained at least one eighth-grade class. Participating schools agreed to have their students assessed. The relative 
bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights that did not 
include a nonresponse adjustment factor, and by their estimated grade 8 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


4.1.3 Logistic Regression Model (TIMSS-8) 


To examine the joint relationship of various characteristics to school nonresponse, the analysis used a 
logistic regression model with participation status as the binary dependent variable and frame 
characteristics as predictor variables. Since public and private schools were modeled together using the 
variables available for all schools, the percentage of students eligible for free or reduced-price lunch was 
not included in the main logistic regression analysis. 


Standard errors and tests of hypotheses for the full model parameter estimates are presented in table F-25 
(with six race/ethnicity variables) and table F-26 (with summed race/ethnicity percentage). Private 
schools are treated as low poverty for the categorical variable poverty level. Only high poverty was a 
significant predictor of school participation in table F-25. The positive parameter estimate indicates that 
relative to schools in the low poverty schools, high poverty schools were somewhat overrepresented 
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among the participating schools. The F test statistic to determine whether the race/ethnicity characteristics 
are simultaneously equal to 0 was 0.51 with a p value of 0.7656, which indicates that no significant 
relationship with participation was detected. Only high poverty was a significant predictor of school 
participation in table F-26. 


Because the percentage of students eligible for free or reduced-price lunch was not included in the main 
logistic regression analysis, a separate analysis with public schools only was conducted. To include free 
or reduced-price lunch eligibility in a model, public schools were modeled separately using a model with 
the summed race/ethnicity percentage and adding the percentage of students eligible for free or reduced- 
price lunch. Since poverty is derived from the percentage of students eligible for free or reduced-price 
lunch, an interaction term’ was also included in the model. Standard errors and tests of hypotheses for the 
full model parameter estimates are presented in table F-27. Only the South region was a significant 
predictor of school participation among public schools only. The positive parameter estimate indicates 
that relative to schools in the West region, schools in the South region were somewhat overrepresented 
among the participating schools. The model with the six race/ethnicity variables is not shown due to 
complex interactions that make the results difficult to interpret. 


Table F-25. Logistic regression model parameters (with six race/ethnicity variables) using the U.S. 
TIMSS eighth-grade original school sample: 2015 


Parameter t test for Ho: 
Parameter estimate Standard error parameter = 0 p value 
Intercept 0.769 0.8091 0.9500 0.3452 
Private school -0.382 0.7493 -0.5102 0.6114 
Central city 0.619 0.7308 0.8475 0.3994 
Suburb 0.290 0.6348 0.4567 0.6492 
Town -0.061 0.7243 -0.0844 0.9329 
Northeast -0.578 0.5183 -1.1158 0.2681 
Midwest 0.344 0.5378 0.6387 0.5250 
South 0.667 0.3959 1.6851 0.0961 
High poverty 0.946 0.4146 2.2806 0.0254 
Total school enrollment 0.001 0.0008 1.5318 0.1298 
Eighth grade enrollment -0.002 0.0017 -1.0990 0.2753 
Black, non-Hispanic -0.010 0.0109 -0.8826 0.3803 
Hispanic -0.013 0.0086 -1.4570 0.1493 
Asian -0.004 0.0179 -0.2066 0.8369 
American Indian or Alaska Native -0.045 0.0640 -0.7042 0.4835 
Hawaiian/Pacific Islander 0.204 0.5711 0.3578 0.7215 
Multiracial -0.087 0.0681 -1.2719 0.2073 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated grade 8 enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


7 The interaction term can be interpreted as indicating whether the marginal effect of a one percentage-point increase in FRPL is diminished or 
amplified for schools above the 50 percent cut point, relative to those below the cut point. 
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Table F-26. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS eighth-grade original school sample: 2015 


Parameter t test for HO: 
Parameter estimate | Standard error parameter = 0 p value 
Intercept 0.507 0.6939 0.7312 0.4669 
Private school -0.242 0.7001 -0.3463 0.7301 
Central city 0.469 0.7124 0.6590 0.5119 
Suburb 0.345 0.6084 0.5673 0.5722 
Town -0.135 0.7134 -0.1898 0.8500 
Northeast -0.417 0.4127 -1.0097 0.3159 
Midwest 0.331 0.4785 0.6914 0.4915 
South 0.609 0.3257 1.8690 0.0655 
High poverty 0.827 0.3471 2.3816 0.0198 
Total school enrollment 0.001 0.0007 1.5088 0.1355 
Eighth grade enrollment -0.001 0.0016 -0.7607 0.4492 
Summed race/ethnicity percentage -0.008 0.0071 -1.1379 0.2588 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Summed race/ethnicity percentage includes Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; 
Hawaiian/Pacific Islander; and two or more races. Black includes African American, and Hispanic includes Latino. Racial categories exclude 
Hispanic origin. Schools were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated 
grade 8 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table F-27. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS eighth-grade original public school sample: 2015 


Parameter Standard t test for HO: 
Parameter estimate error parameter = 0 p value 
Intercept 0.152 0.8474 0.1789 0.8585 
Central city 0.579 0.7301 0.7928 0.4304 
Suburb 0.681 0.6568 1.0367 0.3032 
Town -0.001 0.7412 -0.0013 0.9990 
Northeast -0.229 0.4357 -0.5261 0.6003 
Midwest 0.343 0.4885 0.7029 0.4843 
South 0.746 0.3226 2.3110 0.0236 
High poverty 2.101 1.4743 1.4253 0.1582 
Free or reduced-price lunch eligibility 0.005 0.0139 0.3674 0.7144 
High poverty * free or reduced-price lunch eligibility -0.018 0.0239 -0.7722 0.4424 
Total school enrollment 0.001 0.0007 1.3078 0.1949 
Eighth grade enrollment -0.001 0.0017 -0.5643 0.5742 
Summed race/ethnicity percentage -0.009 0.0081 -1.1061 0.2722 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program. Summed race/ethnicity percentage 
includes Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; Hawaiian/Pacific Islander; and two or more races. Black 
includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were weighted by their school base 
weights that did not include a nonresponse adjustment factor, and by their estimated grade 8 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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4.2 Participating Final Sample with Substitutes 


This section presents the nonresponse bias analysis based on the final sample of 293 eligible schools for 
TIMSS-8 including participating substitute schools. The distribution of the participating final sample was 
compared to the schools in the total eligible final sample. The total eligible final sample includes 
participating final sample plus those original nonrespondents who were not replaced by substitutes. 
School base weights were used for both the eligible sample and the participating schools. Through the use 
of substitute schools, the unweighted school response rate for TIMSS-8 was 84.0 percent after 
replacement, with 246 out of 293 schools participating. The weighted response rate was also 84.0 percent 
after replacement. 


4.2.1 Categorical Variables (TIMSS-8) 


The distribution of participating and eligible schools by the four characteristics is shown in table F-28. 
The Chi-square statistic for Census region and poverty are significant and suggests that there is evidence 
of a relationship with participation in the assessment. In particular, schools in the Northeast were 
underrepresented among participating schools relative to eligible schools (14.6 vs. 17.0 percent, 
respectively), while schools in the South were overrepresented among participating schools (42.5 vs. 38.4 
percent, respectively). High-poverty schools were overrepresented among participating schools (47.5 vs. 
45.1 percent, respectively), and low-poverty schools were underrepresented among participating schools 
(52.5 vs. 54.9 percent, respectively). There are no statistically significant relationships between 
participation status and any of the other characteristics shown in table F-28. 


Table F-28. Percentage distribution of eligible and participating schools in the U.S. TIMSS eighth-grade 
final sample, by selected categorical variables: 2015 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=293) (N=246) Bias bias p value 

School control 0.249 
Public 92.0 91.3 -0.68 -0.007 
Private 8.0 8.7 0.68 0.085 

Locale 0.080 
Central city 30.2 31.0 0.87 0.029 
Suburb 39.9 37.0 -2.89 -0.072 
Town 11.3 11.8 0.53 0.047 
Rural 18.6 20.1 1.49 0.080 

Census region 0.003 
Northeast 17.0 14.6 -2.41 -0.142 
Midwest 21.6 22.1 0.48 0.022 
South 38.4 42.5 4.06 0.106 
West 23.0 20.9 -2.13 -0.093 


See notes at end of table. 
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Table F-28. Percentage distribution of eligible and participating schools in the U.S. TIMSS eighth-grade 
final sample, by selected categorical variables: 2015—Continued 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=293) (N=246) Bias bias p value 

Poverty level 0.026 
High 45.1 47.5 2.48 0.055 
Low 54.9 52.5 -2.48 -0.045 


NOTE: Detail may not sum to totals because of rounding. Census region is the state-based region of the country (see technical notes for state 
listing). For public schools, a high poverty school is defined as one in which 50 percent or more of the students are eligible for participation in the 
FRPL program; all private schools are treated as low poverty schools. Eligible schools contained at least one eighth-grade class. Participating 
schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools 
were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated grade 8 enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


4.2.2 Continuous Variables (TIMSS-8) 


Summary means for each continuous variable for participating and eligible schools are shown in tables 
F-29, F-30, and F-31. One participating school had missing values for race/ethnicity, and this school was 
dropped from the analysis in table F-30. No data on FRPL eligibility was available for private schools, 
and so private schools are not included in the FRPL analysis. Three participating schools had missing 
values for free or reduced-price lunch, and these schools were dropped from the analysis in table F-31. 


There were no statistically significant differences between participating and eligible schools with respect 
to student enrollment, race/ethnicity percentage or free or reduced-price lunch (tables F-29, F-30, and 
F-31). 


Table F-29. Mean enrollment of various characteristics for eligible and participating schools in the U.S. 
TIMSS eighth-grade final sample: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) t test 
Student enrollment (N=293) (N=246) Bias __ Relative bias p value 
Total school 744.1 752.4 8.26 0.011 0.398 
Eighth grade enrollment 242.3 241.7 -0.60 -0.002 0.881 


NOTE: Eligible schools contained at least one eighth-grade class. Participating schools agreed to have their students assessed. The relative bias is 
calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights that did not include a 
nonresponse adjustment factor, and by their estimated grade 8 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide F-31 


Table F-30. Mean percentage of various characteristics for eligible and participating schools in the U.S. 
TIMSS eighth-grade final sample, by race/ethnicity: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) t test 
Race/ethnicity (N=292) (N=245) Bias __ Relative bias p value 
White, non-Hispanic 53.0 53.3 0.28 0.005 0.687 
Black, non-Hispanic 14.9 15.1 0.13 0.009 0.822 
Hispanic 23.8 23.6 -0.15 -0.006 0.794 
Asian 4.9 4.6 -0.27 -0.056 0.155 
American Indian or Alaska 
Native 0.7 0.7 0.01 0.011 0.767 
Hawaiian/Pacific Islander 0.3 0.3 0.03 0.094 0.175 
Multiracial 2.5 2.5 -0.02 -0.007 0.829 


NOTE: Information on race/ethnicity is missing for one of the 246 participating schools in the sample. Black includes African American, and 
Hispanic includes Latino. Racial categories exclude Hispanic origin. Eligible schools contained at least one eighth-grade class. Participating 
schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools 
were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated grade 8 enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table F-31. Mean percentage of students eligible for free or reduced-price lunch, in eligible and 
participating public schools in the U.S. TIMSS eighth-grade final sample: 2015 


Sample schools 


Eligible Participating 
(percent) (percent) t test 
Students (N=267) (N=222) Bias _ Relative bias p value 
Percentage of students 
eligible for free or 
reduced-price lunch 50.8 51.9 1.15 0.023 0.136 


Sa a a a SS ee ee 
NOTE: Information on percentage of students eligible for free or reduced-price lunch is missing for three of the 225 participating public schools 
in the sample. Eligible schools contained at least one eighth-grade class. Participating schools agreed to have their students assessed. The relative 
bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights that did not 

include a nonresponse adjustment factor, and by their estimated grade 8 enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


4.2.3 Logistic Regression Model (TIMSS-8) 


To examine the joint relationship of various characteristics to school nonresponse, the analysis used a 
logistic regression model with participation status as the binary dependent variable and frame 
characteristics as predictor variables. Since public and private schools were modeled together using the 
variables available for all schools, the percentage of students eligible for free or reduced-price lunch was 
not included in the main logistic regression analysis. 


Standard errors and tests of hypotheses for the full model parameter estimates are presented in table F-32 
(with six race/ethnicity variables) and table F-33 (with summed race/ethnicity percentage). Private 
schools are treated as low poverty for the categorical variable poverty level. South region and high 
poverty were significant predictors of school participation in table F-32. The positive parameter estimates 
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indicate that relative to schools in the West region, schools in the South region were somewhat 
overrepresented among the participating schools and that relative to schools in the low poverty schools, 
high poverty schools were somewhat overrepresented among the participating schools. The F test statistic 
to determine whether the race/ethnicity characteristics are simultaneously equal to 0 was 0.85 with a 

p value of 0.517, which indicates no significant relationship was detected with participation. 


South region, high poverty and total enrollment were significant predictors of school participation in table 
F-33. The positive parameter estimate indicates that the total school enrollment in participating schools 
was larger than in all eligible schools. 


Because the percentage of students eligible for free or reduced-price lunch was not included in the main 
logistic regression analysis, a separate analysis with public schools only was conducted. To include free 
or reduced-price lunch eligibility in a model, public schools were modeled separately using a model with 
the summed race/ethnicity percentage and adding the percentage of students eligible for free or reduced- 
price lunch. Since poverty is derived from the percentage of students eligible for free or reduced-price 
lunch, an interaction term was also included in the model. Standard errors and tests of hypotheses for the 
full model parameter estimates are presented in table F-34. Only the South region was a significant 
predictor of school participation among public schools only. The model with the six race/ethnicity 
variables is not shown due to complex interactions that make the results difficult to interpret. 


Table F-32. Logistic regression model parameters (with six race/ethnicity variables) using the U.S. 
TIMSS eighth-grade final school sample: 2015 


Parameter t test for Ho: 
Parameter estimate Standard error parameter = 0 p value 
Intercept 0.421 0.9851 0.4278 0.6700 
Private school 1.217 1.0783 1.1288 0.2626 
Central city 0.014 0.8712 0.0157 0.9875 
Suburb 0.301 0.7794 0.3861 0.7005 
Town -0.617 0.8437 -0.7310 0.4671 
Northeast 0.003 0.6043 0.0052 0.9959 
Midwest 0.99 0.5930 1.6688 0.0993 
South 1.598 0.5179 3.0858 0.0028 
High poverty 1.092 0.4467 2.4438 0.0169 
Total school enrollment 0.002 0.0010 1.8807 0.0639 
Eighth grade enrollment -0.002 0.0018 -1.1740 0.2441 
Black, non-Hispanic -0.017 0.0108 -1.5863 0.1169 
Hispanic -0.01 0.0093 -1.0744 0.2861 
Asian -0.003 0.0177 -0.1774 0.8597 
American Indian or Alaska Native 0.001 0.1163 0.0091 0.9928 
Hawaiian/Pacific Islander 0.751 0.5460 1.3763 0.1728 
Multiracial -0.063 0.0791 -0.7932 0.4302 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated grade 8 enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table F-33. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS eighth-grade final school sample: 2015 


Parameter 
Parameter estimate 
Intercept 0.473 
Private school 1.376 
Central city -0.105 
Suburb 0.339 
Town -0.619 
Northeast -0.164 
Midwest 0.642 
South 1.208 
High poverty 0.951 
Total school enrollment 0.002 
Eighth grade enrollment -0.002 
Summed race/ethnicity percentage -0.009 


Standard error 


0.8548 
0.9714 
0.8521 

0.757 
0.8389 
0.4877 
0.5225 
0.4551 
0.3428 
0.0009 
0.0018 
0.0076 


t test for Ho: 
parameter = 0 


0.5538 
1.4167 
-0.1230 
0.4477 
-0.738 
-0.3371 
1.2292 
2.6552 
2.7734 
2.0249 
-0.8318 
-1.1807 


p value 


0.5814 
0.1607 
0.9024 
0.6557 
0.4628 
0.7370 
0.2229 
0.0097 
0.0070 
0.0464 
0.4082 
0.2415 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Summed race/ethnicity percentage includes Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; 
Hawaiian/Pacific Islander; and two or more races. Black includes African American, and Hispanic includes Latino. Racial categories exclude 
Hispanic origin. Schools were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated 


grade 8 enrollment. 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS) 2015. 


Table F-34. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS eighth-grade final public school sample: 2015 


Parameter 
Parameter estimate 
Intercept -0.272 
Central city -0.137 
Suburb 0.979 
Town -0.629 
Northeast -0.027 
Midwest 0.968 
South 1.490 
High poverty 3.256 
Free or reduced-price lunch eligibility 0.012 
High poverty * free or reduced-price lunch eligibility -0.036 
Total school enrollment 0.002 
Eighth grade enrollment -0.001 
Summed race/ethnicity percentage -0.009 


Standard 
error 


1.0780 
0.8568 
0.8651 
0.8493 
0.5128 
0.5516 
0.4916 
1.7237 
0.0157 
0.0284 
0.0009 
0.0018 
0.0086 


t test for Ho: 
parameter = 0 
-0.2519 
-0.1595 
1.1314 
-0.7400 
-0.0522 
1.7546 
3.0314 
1.8890 
0.7708 
-1.2545 
1.9483 
-0.6718 
-1.0032 


p value 
0.8018 
0.8737 
0.2615 
0.4616 
0.9585 
0.0834 
0.0033 
0.0628 
0.4432 
0.2135 
0.0551 
0.5038 
0.3190 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program. Summed race/ethnicity percentage 
includes Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; Hawaiian/Pacific Islander; and two or more races. Black 
includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were weighted by their school base 
weights that did not include a nonresponse adjustment factor, and by their estimated grade 8 enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS) 2015. 
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4.3 Nonresponse-adjusted Final Sample with Substitutes 


This section presents the nonresponse bias analysis based on the final sample of 293 eligible schools for 
TIMSS-8. The distribution of the participating final sample, including participating substitute schools, 
was compared to the schools in the total eligible final sample, just like the previous section. However, in 
the analyses that follow, school base weights were used for the eligible sample of schools, whereas 
nonresponse-adjusted weights were used for the participating schools. 


4.3.1 Categorical Variables (TIMSS-8) 


The distribution of participating and eligible schools by the four characteristics is shown in table F-35. 
There are no statistically significant relationships between participation status and any of the 
characteristics shown in table F-35. 


Table F-35. Percentage distribution of eligible and participating schools in the U.S. TIMSS eighth-grade 
nonresponse-adjusted sample, by selected categorical variables: 2015 


Sample schools 


Eligible —_— Participating 


(percent) (percent) Relative Chi-square 
School characteristic (N=293) (N=246) Bias bias p value 
School control 0.995 
Public 92.0 92.0 0.00 0.000 
Private 8.0 8.0 0.00 0.000 
Locale 0.466 
Central city 30.2 30.7 0.49 0.016 
Suburb 39.9 38.1 -1.86 -0.046 
Town 11.3 11.5 0.20 0.018 
Rural 18.6 19.8 1.16 0.062 
Census region 1.000 
Northeast 17.0 17.0 0.01 0.000 
Midwest 21.6 21.6 -0.04 -0.002 
South 38.4 38.4 0.02 0.000 
West 23.0 23.0 0.01 0.000 
Poverty level 0.984 
High 45.1 45.0 -0.02 -0.001 
Low 54.9 55.0 0.02 0.000 


NOTE: Detail may not sum to totals because of rounding. Census region is the state-based region of the country (see technical notes for state 
listing). For public schools, a high poverty school is defined as one in which 50 percent or more of the students are eligible for participation in the 
FRPL program; all private schools are treated as low poverty schools. Eligible schools contained at least one eighth-grade class. Participating 
schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools 
were weighted by their school nonresponse adjusted weight and by their estimated grade 8 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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4.3.2 Continuous Variables (TIMSS-8) 


Summary means for each continuous variable for participating and eligible schools are shown in tables 
F-36, F-37 and F-38. One participating school had missing values for race/ethnicity, and this school was 
dropped from the analysis in table F-37. No data on FRPL eligibility was available for private schools, and 
so private schools are not included in the FRPL analysis. Three participating schools had missing values 
for free or reduced-price lunch, and these schools were dropped from the analysis in table F-31. 


There were no statistically significant differences between participating and eligible schools with respect 
to student enrollment, race/ethnicity percentage or free or reduced-price lunch (tables F-36, F-37, and 
F-38). However, the absolute value of the relative bias for Hawaiian/Pacific Islander is greater than 10 
percent, though this is due mostly to the eligible percentage being less than 1.0 percent, as the absolute 
bias is small. 


Table F-36. Mean enrollment of various characteristics for eligible and participating schools in the U.S. 
TIMSS eighth-grade nonresponse-adjusted sample: 2015 


Sample schools 


Eligible Participating 


(mean) (mean) Relative t test 
Student enrollment (N=293) (N=246) Bias bias p value 
Total school 744.1 757.0 12.93 0.017 0.211 
Eighth grade enrollment 242.3 243.1 0.85 0.004 0.851 


NOTE: Eligible schools contained at least one eighth-grade class. Participating schools agreed to have their students assessed. The relative bias is 
calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school nonresponse adjusted weight and 
by their estimated grade 8 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table F-37. Mean percentage of various characteristics for eligible and participating schools in the U.S. 
TIMSS eighth-grade nonresponse-adjusted sample, by race/ethnicity: 2015 


Sample schools 


Eligible Participating 


(mean) (mean) Relative t test 
Race/ethnicity (N=292) (N=245) Bias bias p value 
White, non-Hispanic 53.0 53.8 0.86 0.016 0.246 
Black, non-Hispanic 14.9 14.4 -0.48 -0.032 0.407 
Hispanic 23.8 23.2 -0.59 -0.025 0.317 
Asian 4.9 5.0 0.17 0.035 0.477 
American Indian or 
Alaska Native 0.7 0.7 0.00 0.002 0.966 
Hawaitian/Pacific 
Islander 0.3 0.4 0.06 0.215 0.239 
Multiracial 2.5 2.5 -0.03 -0.012 0.740 


NOTE: Information on race/ethnicity is missing for one of the 246 participating schools in the sample. Black includes African American, and 
Hispanic includes Latino. Racial categories exclude Hispanic origin. Eligible schools contained at least one eighth-grade class. Participating 
schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools 
were weighted by their school nonresponse adjusted weight and by their estimated grade 8 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table F-38. Mean percentage of students eligible for free or reduced-price lunch, in eligible and 
participating public schools in the U.S. TIMSS eighth-grade nonresponse-adjusted sample: 
2015 


Sample schools 


Eligible Participating 
(percent) (percent) Relative t test 
Students (N=267) (N=222) Bias bias p value 
Percentage of students 
eligible for free or 
reduced-price lunch 50.8 50.8 0.01 0.000 0.994 


NOTE: Information on percentage of students eligible for free or reduced-price lunch is missing for three of the 225 participating public schools 
in the sample. Eligible schools contained at least one eighth-grade class. Participating schools agreed to have their students assessed. The relative 
bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school nonresponse adjusted 
weights and by their estimated grade 8 enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


4.4 Summary—Grade 8 


The investigation into nonresponse bias at the school level for the U.S. TIMSS-8 effort shows statistically 
significant relationship between response status and some of the available school characteristics that were 
examined in the analysis. 


For original sample schools, two variables were found to be statistically significantly related to 
participation in the bivariate analysis: Census region (table F-21) and poverty (table F-21). However, the 
absolute value of the relative bias for private schools is greater than 10 percent, which indicates potential 
bias for school control though the absolute bias is relatively small (table F-21). Although each of these 
findings indicates some potential for nonresponse bias, when all of these factors were considered 
simultaneously in a regression analysis, only high poverty was a significant predictor of participation 
(table F-25). The second model showed that only high poverty was a significant predictor of participation 
(table F-26, with summed race/ethnicity percentage). The third model showed only the South region, was 
a significant predictor of school participation among public schools only (table F-27). These results 
suggest that there is some potential for nonresponse bias in the TIMSS-8 participating sample based on 
the characteristics studied. 


For final sample schools (with substitutes), again Census region (table F-28) and poverty (table F-28) 
were found to be statistically significantly related to participation in the bivariate analysis. When all of 
these factors were considered simultaneously in a regression analysis, South region and high poverty were 
significant predictors of participation (table F-32). The second model showed that South region, high 
poverty and total school enrollment were significant predictors of participation (table F-33, with summed 
race/ethnicity percentage). The third model showed South region was a significant predictor of school 
participation among public schools only (table F-34). 


For final sample schools with school nonresponse adjustments applied to the weights, no variables were 
found to be statistically significantly related to participation in the bivariate analysis. However, the 
absolute value of the relative bias for Hawaiian/Pacific Islander is greater than 10 percent, though this is 
due mostly to the eligible percentage being less than 1.0 percent, as the absolute bias is small (table F-37). 
The multivariate regression analysis cannot be conducted after the school nonresponse adjustments are 
applied to the weights. The concept of nonresponse-adjusted weights does not apply to the nonresponding 
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units, and, thus, we cannot conduct an analysis that compares respondents with nonrespondents using 
nonresponse-adjusted weights. 


The results of the analyses are summarized in table F-39. 


Table F-39. Characteristics with p values less than 0.05 and absolute relative bias greater than 10 
percent, U.S. TIMSS eighth-grade schools: 2015 


Additional characteristics with 


Characteristics with p values absolute relative bias greater than 
Analysis less than 0.05 10 percent 
Original sample Census region, poverty private schools 
Regression model a High poverty N/a 
Regression model b High poverty N/a 
Regression model c South region N/a 
Sample with substitutes | Census region, poverty None 
Regression model a South region, high poverty N/a 
Regression model b South region, high poverty, total N/a 
school enrollment 
Regression model c South region N/a 
Nonresponse adjusted None Hawaiian/Pacific Islander 


NOTE: The South region was significant only in the regression model among public schools and not for the entire original sample. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


These results suggest that there is some potential for nonresponse bias in the U.S. TIMSS-8 original 
sample based on the characteristics studied. It also suggests that, while there is some evidence that the use 
of substitute schools has not reduced the potential for bias, it has not added to it substantially. Moreover, 
after the application of school nonresponse adjustments, there is no evidence of resulting potential bias in 
the available frame variables and correlated variables in the final sample. There is less reliance on 
substitution and the nonresponse adjustments in TIMSS-8 than in TIMSS-4 as there are fewer significant 
variables prior to substitution and nonresponse adjustments in TIMSS-8. 
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5. TECHNICAL NOTES 


Description of Variables 


Frame characteristics for public schools were taken from the 2012-13 CCD and, for private schools, from 
the 2011-12 PSS. 


Race/Ethnicity 


Students’ race/ethnicity was obtained through student responses to a two-part question. Students were 
asked first whether they were Hispanic or Latino, and then asked whether they were members of the 
following racial groups: American Indian/Alaska Native; Asian; Black, non-Hispanic; Native Hawaiian or 
other Pacific Islander; or White, non-Hispanic. Two or more races was derived when a student chooses 
more than one of the racial groups. The summed race/ethnicity percentage was derived from summing the 
six race/ethnicities of Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; 
Hawaiian/Pacific Islander and two or more races. 


Locale 


Locale was derived from the urban-centric locale code that is based on the urbanicity of the school 
location. 


5 Central city consists of a large territory inside an urbanized area and inside a principal 
city with population of 250,000 or more, midsize territory inside an urbanized area 
and inside a principal city with a population less than 250,000 and greater than or 
equal to 100,000, or small territory inside an urbanized area and inside a principal city 
with a population less than 100,000. 


a Suburb consists of a large territory outside a principal city and inside an urbanized 
area with population of 250,000 or more, midsize territory outside a principal city and 
inside an urbanized area with a population less than 250,000 and greater than or equal 
to 100,000, or small territory outside a principal city and inside an urbanized area with 
a population less than 100,000. 


cs Town consists of a fringe territory inside an urban cluster that is less than or equal to 
10 miles from an urbanized area, distant territory inside an urban cluster that is more 
than 10 miles and less than or equal to 35 miles from an urbanized area, or remote 
territory inside an urban cluster that is more than 35 miles from an urbanized area. 


7 Rural consists of a fringe census-defined rural territory that is less than or equal to 5 
miles from an urbanized area, as well as rural territory that is less than or equal to 2.5 
miles from an urban cluster, distant census-defined rural territory that is more than 5 
miles but less than or equal to 25 miles from an urbanized area, as well as rural 
territory that is more than 2.5 miles but less than or equal to 10 miles from an urban 
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cluster, or remote census-defined rural territory that is more than 25 miles from an 
urbanized area and is also more than 10 miles from an urban cluster. 


Census Region 


Region is the census region of the United States. Northeast consists of Connecticut, Maine, 
Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, and Vermont. 
Midwest consists of Illinois, Indiana, lowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North 
Dakota, Ohio, South Dakota, and Wisconsin. South consists of Alabama, Arkansas, Delaware, District of 
Columbia, Florida, Georgia, Kentucky, Louisiana, Maryland, Mississippi, North Carolina, Oklahoma, 
South Carolina, Tennessee, Texas, Virginia, and West Virginia. West consists of Alaska, Arizona, 
California, Colorado, Hawaii, Idaho, Montana, Nevada, New Mexico, Oregon, Utah, Washington, and 
Wyoming. 


Percentage of Students Eligible for Free or Reduced-price Lunch 


The proportion of students in a school eligible for the free or reduced-price lunch (FRPL) program, a 
federally assisted meal program under the National School Lunch Act that provides nutritionally 
balanced, low-cost or free lunches to eligible children each school day. The question on the CCD 
questionnaire asked what percentage of students at the school were eligible to receive free or reduced- 
price lunch through the FRPL program around October 1, 2012. It is available only for public schools as 
the NCES Private School Universe Survey (PSS) data do not provide the same information for private 
schools. 


Poverty Level in Public Schools 


The measure of school poverty is based on the percentage of students eligible for FRPL. Schools were 
classified as low poverty if less than 50 percent of the students were eligible for FRPL and as high poverty 
if 50 percent or more of the students were eligible. In the interest of retaining all of the schools and 
students in these analyses, private schools were assumed to be low-poverty schools—that is, they were 
assumed to be schools in which less than 50 percent of students were eligible for FRPL. 
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APPENDIX G: TIMSS ADVANCED 2015 NONRESPONSE 
BIAS ANALYSIS 


1. INTRODUCTION 


The Trends in International Mathematics and Science Study (TIMSS) Advanced is an international 
comparative study of the knowledge, skills, and competencies of advanced math and physics students in 
the domains of mathematics and science. The study was carried out in 9 education systems, including the 
United States. The target population for advanced mathematics is “students in the final year of secondary 
schooling who have taken or were taking courses in advanced mathematics.” The target population for 
physics is “students in the final year of secondary schooling who have taken or were taking courses in 
physics.” The courses that define the target populations have to cover most, if not all, of the advanced 
mathematics and physics topics that were outlined in the TIMSS Advanced 2015 Assessment Frameworks 
(Mullis and Martin 2014). 


The U.S. TIMSS Advanced 2015 study, supported by the National Center for Education Statistics 
(NCES), utilized a two-stage stratified cluster sampling design. The first stage made use of a systematic 
probability-proportionate-to-size technique to select schools. Though efforts were made to secure the 
participation of all schools selected in the first stage, it was anticipated that not all schools would choose 
to participate. Therefore, as each school was selected in the sample, the two neighboring schools in the 
sorted sampling frame (immediately preceding and following it) were designated as replacement schools. 
The sampling frame was sorted by explicit strata and secondarily by implicit strata, so the replacement 
schools were within the same strata as the original school. If an original school refused to participate, the 
first replacement was then contacted. If that school also refused to participate, the second school was then 
contacted. 


The second stage of sampling consisted of selecting students rather than classrooms within sampled 
schools. The methodology was designed specifically for the United States target population and to meet 
international guidelines The TIMSS Advanced 2015 national data collection was fielded in March, April, 
and May 2015. 


There were 348 schools in the original sample for advanced math (hereafter referred to as TIMSS-M). Of 
these 348 sampled schools, 316 were determined to be eligible ! (the eligible original school sample) 
containing at least one advanced math student, and of these, 230 participated (the participating original 
sample) for an initial weighted response rate of 72.2 percent. An additional 11 replacement schools 
participated for a total of 241 participating schools after replacement (the participating final sample). The 
weighted response rate increased to 75.6 percent. The school participation rates for this report are 
summarized in table G-1. 


There were 348 schools in the original sample for advanced physics (hereafter referred to as TIMSS-P). 
Of these 348 sampled schools, 237 were determined to be eligible? (the eligible original school sample) 
containing at least one advanced physics student, and of these, 156 participated (the participating original 
sample) for an initial weighted response rate of 64.9 percent. An additional 9 substitute schools 
participated for a total of 165 participating schools after replacement (the participating final sample). The 
weighted response rate increased to 69.6 percent. 


' Of the 348 original schools selected for the sample, there were 32 excluded or ineligible schools for advanced math. 
? Of the 348 original schools selected for the sample, there were 111 ineligible schools for advanced physics. 
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Table G-1. Selected characteristics for the nonresponse bias analysis of the U.S. TIMSS advanced math 
and physics final school samples: 2015 


Number of participating 
SaiiSale schools Percent 
in Eligible School participation rate School participation rate 
original schools Before After before replacement after replacement 
Subject sample insample replacement replacement Weighted Weighted 
M 348 316 230 241 ADD, 75.6 
P 348 237 156 165 64.9 67.6 


SOURCE: International Association for the Evaluation of Education Achievement (IEA), Trends in International Mathematics and Science Study 
(TIMSS) 2015. 


The weighted student response rate for TIMSS-M was 86.8 percent. The weighted student response rate 
for TIMSS-P was 85.4 percent. 


The National Center for Education Statistics (NCES) standards for assessment surveys stipulate that a 
nonresponse bias analysis is required at any stage of data collection when the weighted unit response rate 
is less than 85 percent (before replacement). Since the U.S. TIMSS Advanced 2015 weighted school 
response rates for both advanced math and physics before replacement are below 85 percent, NCES 
requires an investigation into the potential magnitude of nonresponse bias at the school level in the U.S. 
sample. Since the U.S. TIMSS 2015 weighted student response rates are above 85 percent, a nonresponse 
bias analysis at the student level is not required. The methodology used to investigate nonresponse bias in 
the TIMSS-M and TIMSS-P U.S. samples is provided in chapter 2 of this appendix, and the results are 
provided in chapters 3 and 4. 
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2. METHODOLOGY 


To measure the potential nonresponse bias at the school level, the characteristics of participating schools 
were compared to those of the total eligible sample of schools. This was conducted in a way so that the 
tests of statistical significance that were applied account for the fact that the participating schools are a 
subset of the eligible schools, and not a distinct group. 


The general approach taken involves an analysis in three parts as described below. 


a Analysis of the participating original sample: The distribution for TIMSS-M of the 
participating original school sample (N=230) was compared with that of the total 
eligible original school sample (N=316). The distribution for TIMSS-P of the 
participating original school sample (N=156) was compared with that of the total 
eligible original school sample (N=237). The participating original sample is the 
sample before substitution. In each sample, schools were weighted by their school 
base weights and their estimated advanced math or physics enrollment, excluding any 
nonresponse adjustment factor. The base weight for each original school is the 
reciprocal of its selection probability. 


7 Analysis of the participating final sample with substitutes: The distribution for 
TIMSS-M of the participating final school sample (N=241), which includes 11 
participating substitutes that were used as replacements for nonresponding schools 
from the eligible original sample, was compared to the total eligible final school 
sample (N=316). The distribution for TIMSS-P of the participating final school 
sample (N=165), which includes 9 participating substitutes that were used as 
replacements for nonresponding schools from the eligible original sample, was 
compared to the total eligible final school sample (N=237). The total eligible final 
sample includes the participating final sample plus those original nonrespondents that 
were not replaced by substitutes. Again, schools were weighted by their school base 
weights and their estimated advanced math or physics enrollment for both the eligible 
sample and the participating schools. The base weight for each substitute school is set 
to the base weight of the original school that it replaced. 


a Analysis of the nonresponse adjusted final sample with substitutes: The same sets of 
schools were compared as in the second analysis, but this time, when analyzing the 
participating final schools alone, schools were weighted by their school base weights 
and their estimated advanced math or physics enrollment with school nonresponse 
adjustments applied. The international weighting procedures form nonresponse 
adjustment classes by cross-classifying the explicit and implicit stratification 
variables. 


The first analysis indicates the potential for nonresponse bias that was introduced through school 
nonresponse. The second analysis suggests the remaining potential for nonresponse bias after the 
mitigating effects of substitution have been accounted for. The third analysis indicates the potential for 
bias after accounting for the mitigating effects of both substitution and nonresponse weight adjustments. 
Both the second and third analyses, however, may provide an overly optimistic scenario, resulting from 
the fact that substitution and nonresponse adjustments may correct somewhat for deficiencies in the 
characteristics examined here, but there is no guarantee that they are equally as effective for other 
characteristics and, in particular, for student achievement. 
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Participating TIMSS schools and the total eligible TIMSS school sample were compared on as many 
school sampling frame characteristics as possible that might provide information about the presence of 
nonresponse bias. Comparing frame characteristics between participating schools and the total eligible 
school sample is not an ideal measure of nonresponse bias if the characteristics are unrelated or weakly 
related to more substantive items in the survey; however, often it is the only approach available since 
other data are not available for nonparticipating schools. While the school-level characteristics used in 
these analyses are limited to those available in the sampling frame, each of the variables had a 
demonstrated relationship to achievement in previous TIMSS cycles. 


Frame characteristics for public schools were from the 2012-13 Common Core of Data (CCD) and, for 
private schools, from the 2011-12 Private School Universe Survey (PSS). 


The following categorical variables were available in the sampling frame for all schools: 


rT AP status—indicates whether or not the school had students who took a calculus, 
physics, or both calculus and physics AP test in 2013; 


7 School control—indicates whether the school is under public control (operated by 
publicly elected or appointed officials) or private control (operated by privately 
elected or appointed officials and derives its major source of funds from private 


sources); 
a Locale—urban-centric locale code (i.e., central city, suburb, town, rural); 
7 Census region—Northeast, Midwest, South and West (see Section 5. Technical Notes 


for state listing); and 


m Poverty level*—for public schools, a high poverty school is defined as one in which 
50 percent or more of the students are eligible for participation in the national free and 
reduced-price lunch (FRPL) program, and a low poverty school is defined as one in 
which fewer than 50 percent are eligible. 


The following continuous variables were available in the sampling frame for all schools: 


Estimated number of advanced math or physics students enrolled; 
| Total number of students; and 
n Percentage of students in seven race/ethnicity categories (White, non-Hispanic; Black, 


non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; Hawaiian/Pacific 
Islander and two or more races).* 


An additional continuous variable, the percentage of students eligible to participate in the FRPL program, 
was available only for public schools. The poverty level variable mentioned among the categorical 
variable is the recoded version of this continuous variable.° 


> The sample frame did not contain a direct measure of poverty. No free or reduced-price lunch (FRPL) program data were available for private 
schools, thus all private schools are treated as low-poverty schools. 

4 Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. 

> The continuous variable percentage of students eligible to participate in the FRPL program is missing for private schools; however, private 
schools are treated as low poverty for the categorical variable poverty level. 
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For categorical variables, the distribution of frame characteristics for participating schools was compared 
with the distribution for all eligible schools. The hypothesis of independence between the characteristic 
and participation status was tested using a Rao-Scott modified Chi-square statistic at the 5 percent level 
(Rao and Thomas 2003). For continuous variables, summary means were calculated and the difference 
between means was tested using a ¢ test. The p values for the tests are presented in the tables that follow. 
The statistical significance of differences between participants and the total eligible sample is identical to 
that which would result from comparing participants and nonparticipants, since all significance tests 
account for the fact that the participants are a subset of the full sample. The bias and relative bias are also 
shown in each table. The bias is calculated as the difference between the respective estimates for the 
participants and the eligible sample. The relative bias is calculated as the bias divided by the estimate 
from the eligible sample. The relative bias is a measure of the size of the bias compared to the eligible 
sample estimate. 


In addition to these tests, logistic regression models were used to provide a multivariate analysis that 
examined the conditional independence of these school characteristics as predictors of participation. The 
logistic regression compared frame characteristics for participating schools with non-participating schools 
which is effectively the same as comparing to the eligible sample as in the bivariate analysis. It may be 
that only one or two variables are actually related to participation status. However, if these variables are 
also related to the other variables examined in the analyses, then other variables, which are not related to 
participation status, will appear as significant in simple bivariate tables. Dummy variables were created 
for each component of the categorical variables so that each component was included separately. The last 
component of each categorical variable is used as the reference category. The p value of a dummy 
variable indicates whether there is a significant difference at the 5 percent level from the effect of the 
(omitted) reference category. It is not possible to include all the frame characteristics in a single model 
because the seven race/ethnicity variables are linearly dependent (1.e., they sum up to 100 percent for 
every school). Therefore, two models were used. In the first model, six race/ethnicities (Black, non- 
Hispanic; Hispanic; Asian; American Indian or Alaska Native; Hawaiian/Pacific Islander; and two or 
more races) were included in the model with “percentage White, non-Hispanic” as the omitted category. 
In addition, an F test was used to determine whether the parameter estimates of these six characteristics 
were simultaneously equal to zero. In the second model, the summed percentage of the six race/ethnicities 
(Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; Hawaiian/Pacific Islander; 
and two or more races) replaced the six race/ethnicity variables with “percentage White, non-Hispanic” 
again as the omitted category. The second model permits the analysis of differences in the percentages of 
White, non-Hispanic students, which is not possible in the first model. All other frame characteristics 
were included in both models. 


The logistic regression was performed using WesVar® (Westat 2007) and replicate weights to properly 


account for the complex sample design. The JK2 method was used to create the replicate weights 
(Westat 2007). 
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3. RESULTS—TIMSS ADVANCED MATH 


3.1 Original Respondent Sample 


This section presents the nonresponse bias analysis based on the original sample of 316 eligible schools 
for TIMSS-M. The distribution of the participating original sample was compared to the schools in the 
total eligible original sample. School base weights were used for both the eligible sample and the 
participating schools. The unweighted school response rate for TIMSS-M was 72.8 percent before 
replacement, with 230 out of 316 schools participating. The weighted response rate was 72.2 percent 
before replacement. 


3.1.1 Categorical Variables (TIMSS-M) 


The distribution of participating and eligible schools by the four characteristics is shown in table G-2. 
There are no statistically significant relationships between participation status and any of the 
characteristics shown in table G-2. However, the absolute value of the relative bias for rural is greater 
than 10 percent, which indicates potential bias for locale (table G-2). 


Table G-2. Percentage distribution of eligible and participating schools in the U.S. TIMSS advanced 
math original sample, by selected categorical variables: 2015 


Sample schools 


Eligible Participating 
School (percent) (percent) Relative Chi-square 
characteristic (N=316) (N=230) Bias bias p value 
AP status 0.917 
Non-AP 
school 15.4 15.6 0.20 0.002 
AP school 84.6 84.4 -0.20 -0.002 
School control 0.780 
Public 87.0 87.5 0.48 0.006 
Private 13.0 12.5 -0.48 -0.037 
Locale 0.066 
Central city 27.8 26.1 -1.70 -0.061 
Suburb 44.4 41.4 -2.95 -0.066 
Town 8.4 9.0 0.62 0.074 
Rural 19.4 23.5 4.03 0.207 
Census region 0.653 
Northeast 18.3 16.7 -1.60 -0.087 
Midwest 24.1 25.6 1.46 0.061 
South 33.4 34.9 1.51 0.045 
West 24.2 22.8 -1.37 -0.057 


See notes at end of table. 
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Table G-2. Percentage distribution of eligible and participating schools in the U.S. TIMSS advanced 
math original sample, by selected categorical variables: 2015—Continued 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=316) (N=230) Bias bias p value 

Poverty level 0.345 
High 21.7 23.4 1.62 0.074 
Low 78.3 76.6 -1.62 -0.021 


NOTE: Detail may not sum to totals because of rounding. AP status indicates whether or not the school has students who took a relevant AP test 
in 2013. Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low-poverty schools. Eligible schools contained at least one advanced math student. Participating schools agreed to have their students assessed. 
The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights 
that did not include a nonresponse adjustment factor, and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


3.1.2. Continuous Variables (TIMSS-M) 


Summary means for each continuous variable for participating and eligible schools are shown in 
tables G 3, G-4 and G-5. No data on FRPL eligibility were available for private schools, so private 
schools are not included in the FRPL analysis. Three eligible schools had missing values for free or 
reduced-price lunch, and these schools were dropped from the analysis in table G-5. 


There were no statistically significant differences between participating and eligible schools with respect 
to student enrollment, race/ethnicity percentage or free or reduced-price lunch (tables G-3, G-4 and G-5). 
However, the absolute value of the relative bias for multiracial is greater than 10 percent (table G-4). 
Though for multiracial this is due mostly to the eligible percentage being less than 3.0 percent, as the 
absolute bias is small. 


Table G-3. Mean enrollment of various characteristics for eligible and participating schools in the U.S. 
TIMSS advanced math original sample: 2015 


Sample schools 
Eligible Participating 


(mean) (mean) Relative t test 
Student enrollment (N=316) (N=230) Bias bias p value 
Total school 1,418.2 1,364.8 -53.38 -0.038 0.201 
Advanced math 67.3 66.2 -1.06 -0.016 0.724 


NOTE: Eligible schools contained at least one advanced math student. Participating schools agreed to have their students assessed. The relative 
bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights that did not 
include a nonresponse adjustment factor, and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table G-4. Mean percentage of various characteristics for eligible and participating schools in the U.S. 
TIMSS advanced math original sample, by race/ethnicity: 2015 


Sample schools 


Eligible Participating 


(mean) (mean) Relative t test 
Race/ethnicity (N=316) (N=230) Bias bias p value 
White, non-Hispanic 58.9 59.3 0.49 0.008 0.929 
Black, non-Hispanic 10.5 9.5 -1.01 -0.096 0.685 
Hispanic 19.6 20.2 0.59 0.030 0.261 
Asian 7.7 7.9 0.14 0.018 0.548 
American Indian or 
Alaska Native 0.5 0.6 0.03 0.059 0.789 
Hawaiian/Pacific 
Islander 0.2 0.2 0.00 0.021 0.507 
Multiracial 25 2.3 -0.25 -0.100 0.834 


NOTE: Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Eligible schools contained at 
least one advanced math student. Participating schools agreed to have their students assessed. The relative bias is calculated as the bias divided by 
the estimate from the eligible sample. Schools were weighted by their school base weights that did not include a nonresponse adjustment factor, 
and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table G-5. Mean percentage of students eligible for free or reduced-price lunch, in eligible and 
participating public schools in the U.S. TIMSS advanced math original sample: 2015 


Sample schools 


Eligible Participating 


(percent) (percent) Relative t test 
Students (N=292) (N=218) Bias bias p value 


Percentage of students 
eligible for free or 
reduced-price 
lunch 34.5 35.7 1.19 0.035 0.286 


NOTE: Information on percentage of students eligible for free or reduced-price lunch is missing for three of the 295 eligible public schools in the 
sample. Eligible schools contained at least one advanced math student. Participating schools agreed to have their students assessed. The relative 
bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights that did not 
include a nonresponse adjustment factor, and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


3.1.3 Logistic Regression Model (TIMSS-M) 


To examine the joint relationship of various characteristics to school nonresponse, the analysis used a 
logistic regression model with participation status as the binary dependent variable and frame 
characteristics as predictor variables. Since public and private schools were modeled together using the 
variables available for all schools, the percentage of students eligible for free or reduced-price lunch was 
not included in the main logistic regression analysis. 


Standard errors and tests of hypotheses for the full model parameter estimates are presented in table G-6 


(with six race/ethnicity variables) and table G-7 (with summed race/ethnicity percentage). Private schools 
are treated as low poverty for the categorical variable poverty level. None of the parameter estimates are 
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significant in table G-6. The F test statistic to determine whether the race/ethnicity characteristics are 
simultaneously equal to 0 was 0.39 with a p value of 0.852, which indicates that no significant 
relationship with participation was detected. None of the parameter estimates are significant in table G-7. 


Because the percentage of students eligible for free or reduced-price lunch was not included in the main 
logistic regression analysis, a separate analysis with public schools only was conducted. To include free 
or reduced-price lunch eligibility in a model, public schools were modeled separately using a model with 
the summed race/ethnicity percentage and adding the percentage of students eligible for free or reduced- 
price lunch. Since poverty is derived from the percentage of students eligible for free or reduced-price 
lunch, an interaction term® was also included in the model. Standard errors and tests of hypotheses for the 
full model parameter estimates are presented in table G-8. There were no significant predictors of school 
participation among public schools only in table G-8. 


Table G-6. Logistic regression model parameters (with six race/ethnicity variables) using the U.S. 
TIMSS advanced math original school sample: 2015 


Parameter Standard t test for HO: 
Parameter estimate error parameter = 0 p value 
Intercept 1.445 0.8443 1.7120 0.0910 
Non-AP school -0.612 0.5749 -1.0648 0.2904 
Private school -0.399 0.5563 -0.7164 0.4760 
Central city 0.008 0.6391 0.0127 0.9899 
Suburb 0.970 0.6457 1.5028 0.1371 
Town -0.117 0.6471 -0.1803 0.8574 
Northeast -0.361 0.7148 -0.5051 0.6149 
Midwest 0.570 0.7100 0.8022 0.4250 
South 0.553 0.5752 0.9620 0.3391 
High poverty 0.436 0.4544 0.9605 0.3399 
Total school enrollment 0.000 0.0003 -1.3571 0.1788 
Advanced math enrollment 0.002 0.0036 0.6314 0.5297 
Black, non-Hispanic -0.017 0.0132 -1.2514 0.2147 
Hispanic 0.004 0.0118 0.3753 0.7085 
Asian 0.011 0.0202 0.5324 0.5961 
American Indian or Alaska Native 0.097 0.6267 0.1541 0.8779 
Hawaiian/Pacific Islander 0.150 0.5790 0.2596 0.7959 
Multiracial -0.123 0.0717 -1.7122 0.0910 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated advanced math enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


° The interaction term can be interpreted as indicating whether the marginal effect of a one percentage-point increase in FRPL is diminished or 
amplified for schools above the 50 percent cut point, relative to those below the cut point. 
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Table G-7. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS advanced math original school sample: 2015 


Parameter Standard t test for HO: 
Parameter estimate error parameter = 0 p value 
Intercept 1.430 0.6785 2.1070 0.0385 
Non-AP school -0.516 0.5745 -0.8988 0.3717 
Private school -0.496 0.4883 -1.0158 0.3130 
Central city -0.385 0.5810 -0.6634 0.5091 
Suburb 0.798 0.6581 1.2121 0.2293 
Town -0.376 0.5942 -0.6323 0.5291 
Northeast -0.271 0.5937 -0.4559 0.6498 
Midwest 0.280 0.5733 0.4889 0.6263 
South 0.291 0.4824 0.6040 0.5477 
High poverty 0.656 0.4998 1.3134 0.1931 
Total school enrollment 0.000 0.0003 -1.3154 0.1924 
Advanced math enrollment 0.003 0.0031 0.9336 0.3535 
Summed race/ethnicity percentage -0.002 0.0079 -0.2164 0.8293 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Summed race/ethnicity percentage includes Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; 
Hawaiian/Pacific Islander; and two or more races. Black includes African American, and Hispanic includes Latino. Racial categories exclude 
Hispanic origin. Schools were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated 


advanced math enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS) 2015. 


Table G-8. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS advanced math original public school sample: 2015 


Parameter Standard t test for HO: 

Parameter estimate error parameter = 0 p value 
Intercept 1.720 0.8379 2.0525 0.0436 
Non-AP school -0.287 0.6941 -0.4134 0.6805 
Central city -0.075 0.6511 -0.1146 0.9091 
Suburb 0.378 0.7019 0.5388 0.5916 
Town -0.069 0.6416 -0.1070 0.9151 
Northeast -0.583 0.6494 -0.8984 0.3718 
Midwest 0.079 0.6074 0.1296 0.8973 
South 0.617 0.5256 1.1748 0.2438 
High poverty 0.693 2.5839 0.2683 0.7892 
Free or reduced-price lunch eligibility 0.021 0.0146 1.4055 0.1640 
High poverty * free or reduced-price 

lunch eligibility -0.010 0.0408 -0.2454 0.8068 
Total school enrollment 0.000 0.0003 -1.0728 0.2868 
Advanced math enrollment 0.003 0.0032 0.9206 0.3602 
Summed race/ethnicity percentage -0.012 0.0074 -1.5961 0.1147 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program. Summed race/ethnicity percentage 
includes Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; Hawaiian/Pacific Islander; and two or more races. Black 
includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were weighted by their school base 
weights that did not include a nonresponse adjustment factor, and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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3.2 Participating Final Sample with Substitutes 


This section presents the nonresponse bias analysis based on the final sample of 316 eligible schools for 
TIMSS-M including participating substitute schools. The distribution of the participating final sample 
was compared to the schools in the total eligible final sample. The total eligible final sample includes 
participating final sample plus those original nonrespondents who were not replaced by substitutes. 
School base weights were used for both the eligible sample and the participating schools. Through the use 
of substitute schools, the unweighted school response rate for TIMSS-M was 76.3 percent after 
replacement, with 241 out of 316 schools participating. The weighted response rate was 75.6 percent after 
replacement. 


3.2.1 Categorical Variables (TIMSS-M) 


The distribution of participating and eligible schools by the four characteristics is shown in table G-9. 
There are no statistically significant relationships between participation status and any of the 
characteristics shown in table G-9. However, the absolute value of the relative bias for town, rural, and 
Northeast region is greater than 10 percent, which indicates potential bias for locale and Census region 
(table G-9). 


Table G-9. Percentage distribution of eligible and participating schools in the U.S. TIMSS advanced 
math final sample, by selected categorical variables: 2015 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=316) (N=241) Bias bias p value 

AP status 0.843 
Non-AP school 15.1 14.8 -0.37 -0.024 
AP school 84.9 85.2 0.37 0.004 

School control 0.905 
Public 87.5 87.7 0.20 0.002 
Private 12.5 12.3 -0.20 -0.016 

Locale 0.050 
Central city 28.0 27.0 -0.93 -0.033 
Suburb 43.5 39.9 -3.64 -0.084 
Town 8.9 10.1 1.11 0.124 
Rural 19.6 23.0 3.47 0.177 

Census region 0.397 
Northeast 18.0 15.9 -2.19 -0.121 
Midwest 24.2 25.9 1.71 0.071 
South 34.2 36.3 2.10 0.061 
West 23.5 21.9 -1.63 -0.069 


See notes at end of table. 
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Table G-9. Percentage distribution of eligible and participating schools in the U.S. TIMSS advanced 
math final sample, by selected categorical variables: 2015—Continued 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=316) (N=241) Bias bias p value 

Poverty level 0.399 
High 21.1 22:5 1.39 0.066 
Low 78.9 TAS -1.39 -0.018 


NOTE: Detail may not sum to totals because of rounding. AP status indicates whether or not the school has students who took a relevant AP test 
in 2013. Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Eligible schools contained at least one advanced math student. Participating schools agreed to have their students assessed. 
The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights 
that did not include a nonresponse adjustment factor, and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


3.2.2 Continuous Variables (TIMSS-M) 


Summary means for each continuous variable for participating and eligible schools are shown in 

tables G-10, G-11, and G-12. No data on FRPL eligibility was available for private schools, and so 
private schools are not included in the FRPL analysis. Three eligible schools had missing values for free 
or reduced-price lunch, and these schools were dropped from the analysis in table G-12. 


There were no statistically significant differences between participating and eligible schools with respect 
to student enrollment, race/ethnicity percentage or free or reduced-price lunch (tables G-10, G-11, and 
G-12). 


Table G-10. Mean enrollment of various characteristics for eligible and participating schools in the U.S. 
TIMSS advanced math final sample: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) Relative t test 
Student enrollment (N=3 16) (N=241) Bias bias p value 
Total school 1,420.6 1,381.3 -39.27 -0.028 0.315 
Advanced math 69.0 68.2 -0.80 -0.012 0.775 


NOTE: Eligible schools contained at least one advanced math student. Participating schools agreed to have their students assessed. The relative 
bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights that did not 
include a nonresponse adjustment factor, and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table G-11. Mean percentage of various characteristics for eligible and participating schools in the U.S. 
TIMSS advanced math final sample, by race/ethnicity: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) Relative t test 
Race/ethnicity (N=316) (N=241) Bias bias p value 
White, non-Hispanic 59.5 60.3 0.88 0.015 0.776 
Black, non-Hispanic 10.3 9.3 -1.02 -0.099 0.457 
Hispanic 19.2 19.5 0.34 0.017 0.246 
Asian 7.8 Tel -0.02 -0.003 0.722 
American Indian or Alaska 
Native 0.5 0.5 0.02 0.035 0.967 
Hawaiian/Pacific Islander 0.2 0.2 0.00 0.003 0.672 
Multiracial 2.6 2.4 -0.19 -0.074 0.974 


NOTE: Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Eligible schools contained at 
least one advanced math student. Participating schools agreed to have their students assessed. The relative bias is calculated as the bias divided by 
the estimate from the eligible sample. Schools were weighted by their school base weights that did not include a nonresponse adjustment factor, 
and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table G-12. Mean percentage of students eligible for free or reduced-price lunch, in eligible and 
participating public schools in the U.S. TIMSS advanced math final sample: 2015 


Sample schools 


Eligible _—_— Participating 


(percent) (percent) t test 

Students (N=292) (N=228) Bias _ Relative bias p value 
Percentage of students eligible for 

free or reduced-price lunch 29.5 31.3 1.84 0.063 0.284 


Da a a 8 is) 
NOTE: Information on percentage of students eligible for free or reduced-price lunch is missing for three of the 295 eligible public schools in the 

sample. Eligible schools contained at least one advanced math student. Participating schools agreed to have their students assessed. The relative 

bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights that did not 

include a nonresponse adjustment factor, and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 

Study (TIMSS) 2015. 


3.2.3 Logistic Regression Model (TIMSS-M) 


To examine the joint relationship of various characteristics to school nonresponse, the analysis used a 
logistic regression model with participation status as the binary dependent variable and frame 
characteristics as predictor variables. Since public and private schools were modeled together using the 
variables available for all schools, the percentage of students eligible for free or reduced-price lunch was 
not included in the main logistic regression analysis. 


Standard errors and tests of hypotheses for the full model parameter estimates are presented in table G-13 
(with six race/ethnicity variables) and table G-14 (with summed race/ethnicity percentage). Private 
schools are treated as low poverty for the categorical variable poverty level. None of the parameter 
estimates are significant in table G-13. The F test statistic to determine whether the race/ ethnicity 
characteristics are simultaneously equal to 0 was 0.52 with a p value of 0.763, which indicates no 
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significant relationship was detected with participation. None of the parameter estimates are significant in 
table G-14. 


Because the percentage of students eligible for free or reduced-price lunch was not included in the main 
logistic regression analysis, a separate analysis with public schools only was conducted. To include free 
or reduced-price lunch eligibility in a model, public schools were modeled separately using a model with 
the summed race/ethnicity percentage and adding the percentage of students eligible for free or reduced- 
price lunch. Since poverty is derived from the percentage of students eligible for free or reduced-price 
lunch, an interaction term was also included in the model. Standard errors and tests of hypotheses for the 
full model parameter estimates are presented in table G-15. None of the parameter estimates are 
significant in table G-15. The model with the six race/ethnicity variables is not shown due to complex 
interactions that make the results difficult to interpret. 


Table G-13. Logistic regression model parameters (with six race/ethnicity variables) using the U.S. 
TIMSS advanced math final school sample: 2015 


Parameter Standard t test for Ho: 
Parameter estimate error parameter = 0 p value 
Intercept 1.731 0.8685 1.9925 0.0500 
Non-AP school -0.794 0.5734 -1.3842 0.1704 
Private school -0.284 0.6701 -0.4242 0.6727 
Central city -0.267 0.6503 -0.4099 0.6831 
Suburb 0.642 0.6384 1.0058 0.3178 
Town -0.550 0.6528 -0.8420 0.4024 
Northeast -0.302 0.7282 -0.4147 0.6796 
Midwest 0.766 0.7891 0.9709 0.3347 
South 0.798 0.6303 1.2657 0.2095 
High poverty 0.611 0.4767 1.2809 0.2042 
Total school enrollment 0.000 0.0003 -1.0041 0.3186 
Advanced math enrollment 0.001 0.0035 0.4188 0.6765 
Black, non-Hispanic -0.020 0.0130 -1.5662 0.1215 
Hispanic 0.002 0.0122 0.1804 0.8574 
Asian 0.008 0.0198 0.3863 0.7003 
American Indian or Alaska Native 0.071 0.4602 0.1534 0.8785 
Hawaiian/Pacific Islander 0.191 0.6118 0.3121 0.7558 
Multiracial -0.103 0.0725 -1.4173 0.1605 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated advanced math enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table G-14. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS advanced math final school sample: 2015 


Parameter Standard t test for Ho: 
Parameter estimate error parameter = 0 p value 
Intercept 1.798 0.7156 25121 0.0142 
Non-AP school -0.705 0.5721 -1.2322 0.2217 
Private school -0.384 0.6072 -0.6321 0.5293 
Central city -0.610 0.6226 -0.9794 0.3305 


See notes at end of table. 
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Table G-14. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS advanced math final school sample: 2015—Continued 


Parameter Standard 
Parameter estimate error 
Suburb 0.505 0.6771 
Town -0.763 0.6158 
Northeast -0.298 0.604 
Midwest 0.402 0.619 
South 0.468 0.5384 
High poverty 0.797 0.5313 
Total school enrollment 0.000 0.0003 
Advanced math enrollment 0.002 0.0031 
Summed race/ethnicity percentage -0.005 0.008 


t test for Ho: 
parameter = 0 


0.7462 
-1.2393 
-0.4931 

0.6498 

0.8691 

1.5002 
-0.8981 

0.6915 
-0.5842 


p value 


0.4579 
0.2191 
0.6234 
0.5178 
0.3876 
0.1378 
0.3720 
0.4914 
0.5608 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS) 2015. 


Table G-15. Logistic regression model parameters (with summed race/ethnicity percentage) using the 


U.S. TIMSS advanced math final public school sample: 2015 


Parameter 
Parameter estimate 
Intercept 1.473 
Non-AP school -0.478 
Central city -0.465 
Suburb -0.063 
Town -0.494 
Northeast -0.603 
Midwest 0.249 
South 0.677 
High poverty 1.341 
Free or reduced-price lunch eligibility 0.023 
High poverty * free or reduced-price lunch 
eligibility -0.020 
Total school enrollment 0.000 
Advanced math enrollment 0.002 
Summed race/ethnicity percentage -0.014 


Standard 
error 
0.7952 
0.6937 
0.6319 
0.7128 
0.6363 
0.6337 
0.6289 
0.5602 
2.9505 
0.0157 


0.0454 
0.0003 
0.0033 
0.0073 


t test for HO: 
parameter = 0 
1.8525 
-0.6892 
-0.7365 
-0.0878 
-0.7764 
-0.9517 
0.3952 
1.2086 
0.4545 
1.4944 


-0.4457 
-0.7083 

0.6126 
-1.8487 


p value 
0.0679 
0.4928 
0.4637 
0.9303 
0.4400 
0.3443 
0.6938 
0.2306 
0.6508 
0.1393 


0.6571 
0.4809 
0.5420 
0.0684 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program. Black includes African American, 
and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were weighted by their school base weights that did not include 


a nonresponse adjustment factor, and by their estimated advanced math enrollment. 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS) 2015. 


3.3 Nonresponse-adjusted Final Sample with Substitutes 


This section presents the nonresponse bias analysis based on the final sample of 316 eligible schools for 
TIMSS-M. The distribution of the participating final sample, including participating substitute schools, 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide 


G-20 


was compared to the schools in the total eligible final sample, just like the previous section. However, in 
the analyses that follow, school base weights were used for the eligible sample of schools, whereas 
nonresponse-adjusted weights were used for the participating schools. 


3.3.1 Categorical Variables (TIMSS-M) 


The distribution of participating and eligible schools by the four characteristics is shown in table G-16. 
There are no statistically significant relationships between participation status and any of the 
characteristics shown in table G-16. However, the absolute value of the relative bias for town and rural is 
greater than 10 percent, which indicates potential bias for locale (table G-16). 


Table G-16. Percentage distribution of eligible and participating schools in the U.S. TIMSS advanced 
math nonresponse-adjusted sample, by selected categorical variables: 2015 


Sample schools 
Eligible Participating 


(percent) (percent) Relative Chi-square 
School characteristic (N=316) (N=241) Bias bias p value 
AP status 0.795 
Non-AP school 15.1 14.7 -0.49 -0.032 
AP school 84.9 85.3 0.49 0.006 
School control 0.559 
Public 87.5 86.3 -1.17 -0.013 
Private 12.5 13.7 1.17 0.094 
Locale 0.069 
Central city 28.0 26.8 -1.14 -0.041 
Suburb 43.5 40.1 -3.50 -0.080 
Town 8.9 9.9 0.92 0.103 
Rural 19.6 23.3 3.72 0.190 
Census region 0.865 
Northeast 18.0 18.9 0.83 0.046 
Midwest 24.2 25.0 0.83 0.034 
South 34.2 32.4 -1.77 -0.052 
West 23:5 23.7 0.12 0.005 
Poverty level 0.446 
High 21.1 22.4 1.29 0.061 
Low 78.9 77.6 -1.29 -0.016 


NOTE: Detail may not sum to totals because of rounding. AP status indicates whether or not the school has students who took a relevant AP test 
in 2013. Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Eligible schools contained at least one advanced math student. Participating schools agreed to have their students assessed. 
The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school nonresponse 
adjusted weight and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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3.3.2 Continuous Variables (TIMSS-M) 


Summary means for each continuous variable for participating and eligible schools are shown in 

tables G-17, G-18 and G-19. No data on FRPL eligibility was available for private schools, and so private 
schools are not included in the FRPL analysis. Three eligible schools had missing values for free or 
reduced-price lunch, and these schools were dropped from the analysis in table G-19. 


There were no statistically significant differences between participating and eligible schools with respect 
to student enrollment, race/ethnicity percentage or free or reduced-price lunch (tables G-17, G-18, and 
G-19). However, the absolute values of the relative bias for Black, non-Hispanic is greater than 

10 percent (table G-18). 


Table G-17. Mean enrollment of various characteristics for eligible and participating schools in the U.S. 
TIMSS advanced math nonresponse-adjusted sample: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) Relative t test 
Student enrollment (N=316) (N=241) Bias bias p value 
Total school 1,420.6 1,362.4 -58.12 -0.041 0.152 
Advanced math 69.0 66.8 -2.18 -0.032 0.401 


NOTE: Eligible schools contained at least one advanced math student. Participating schools agreed to have their students assessed. The relative 
bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school nonresponse adjusted 
weight and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table G-18. Mean percentage of various characteristics for eligible and participating schools in the U.S. 
TIMSS advanced math nonresponse-adjusted sample, by race/ethnicity: 2015 


Sample schools 


Eligible Participating 

(mean) (mean) Relative t test 
Race/ethnicity (N=316) (N=241) Bias bias p value 
White, non-Hispanic 59.5 60.7 1.22 0.021 0.521 
Black, non-Hispanic 10.3 8.9 -1.46 -0.142 0.309 
Hispanic 19.2 19.4 0.24 0.012 0.094 
Asian 7.8 8.0 0.21 0.027 0.804 

American Indian or Alaska 
Native 0.5 0.5 0.01 0.016 0.707 
Hawaiian/Pacific Islander 0.2 0.2 0.01 0.028 0.847 
Multiracial 2.6 2.3 -0.22 -0.086 0.792 


NOTE: Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Eligible schools contained at 
least one advanced math student. Participating schools agreed to have their students assessed. The relative bias is calculated as the bias divided by 
the estimate from the eligible sample. Schools were weighted by their school nonresponse adjusted weight and by their estimated advanced math 
enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table G-19. Mean percentage of students eligible for free or reduced-price lunch, in eligible and 
participating public schools in the U.S. TIMSS advanced math nonresponse-adjusted 
sample: 2015 


Sample schools 
Eligible _—_ Participating 


(percent) (percent) t test 

Students (N=292) (N=228) Bias _ Relative bias p value 
Percentage of students eligible for 

free or reduced-price lunch 29.5 30.9 1.43 0.049 0.420 


ee Se ee ee ee ee eee 
NOTE: Information on percentage of students eligible for free or reduced-price lunch is missing for three of the 295 eligible public schools in the 

sample. Eligible schools contained at least one advanced math student. Participating schools agreed to have their students assessed. The relative 

bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school nonresponse adjusted 

weights and by their estimated advanced math enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 

Study (TIMSS) 2015. 


3.4 Summary—Advanced Math 


The investigation into nonresponse bias at the school level for the U.S. TIMSS-M effort shows no 
statistically significant relationships between response status and all of the available school characteristics 
that were examined in the analysis. 


For original sample schools, no variables were found to be statistically significantly related to 
participation in the bivariate analysis. However, the absolute value of the relative bias for rural and 
multiracial is greater than 10 percent (tables G-2 and G-4, respectively), which indicates potential bias 
even though no statistically significant relationship was detected. Although each of these findings 
indicates some potential for nonresponse bias, when all of these factors were considered simultaneously 
in a regression analysis, none of the parameter estimates are significant predictors of participation (tables 
G-6 and G-7, with summed race/ethnicity percentage, G-8). None of the parameter estimates are 
significant predictors of participation among public schools only (table G-8). 


For final sample schools (with substitutes), no variables were found to be statistically significantly related 
to participation in the bivariate analysis. However, the absolute value of the relative bias for town, rural, 
and Northeast region is greater than 10 percent (table G-9), which indicates potential bias even though no 
statistically significant relationship was detected. When all of these factors were considered 
simultaneously in a regression analysis, none of the parameter estimates are significant predictors of 
participation (tables G-13 and G-14, with summed race/ethnicity percentage). None of the parameter 
estimates are significant predictors of participation among public schools only (table G-15). 


For final sample schools with school nonresponse adjustments applied to the weights, no variables were 
found to be statistically significantly related to participation in the bivariate analysis. However, the 
absolute value of the relative bias for town, rural, and Black, non-Hispanic is greater than 10 percent, 
(tables G-16 and G-18), which indicates potential bias even though no statistically significant relationship 
was detected. The multivariate regression analysis cannot be conducted after the school nonresponse 
adjustments are applied to the weights. The concept of nonresponse-adjusted weights does not apply to 
the nonresponding units, and, thus, we cannot conduct an analysis that compares respondents with 
nonrespondents using nonresponse-adjusted weights. 


The results of the analyses are summarized in table G-20. 
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Table G-20. Characteristics with p values less than 0.05 and absolute relative bias greater than 10 
percent, U.S. TIMSS advanced math schools: 2015 


Characteristics with p values Additional characteristics with absolute 

Analysis less than 0.05 relative bias greater than 10 percent 
Gugidal sample ane Rural, multiracial 

Regression model a None t 

Regression model b None t 

Regression model c None t 
Sample with substitutes None Town, Hust, Neribeastrerion 

Regression model a None t 

Regression model b None t 

Regression model c None t 
Nonresponse adjusted None Town, rural, Black non-Hispanic 
+ Not applicable. 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Note that the standard errors for both advanced math and physics are generally much larger than for 
grades 4 and 8. This means some large biases could have been significant had the standard errors been in 
line with grades 4 and 8. This was likely due to having a measure of size for school sampling (grade 12 
enrollment) that was not a good measure of the true number of advanced students. Additionally, the 
estimated advanced math and physics enrollments (used as the measure of school size for conducting 
these nonresponse bias analyses) were not always a good measure of the true number of advanced 
students. 


These results suggest that there is little potential for nonresponse bias in the U.S. TIMSS-M original 
sample based on the characteristics studied. It also suggests that, while there is some evidence that the use 
of substitute schools has not reduced the potential for bias, it has not added to it substantially. Moreover, 
after the application of school nonresponse adjustments, there is little evidence of resulting potential bias 
in the available frame variables and correlated variables in the final sample. Given there is limited 
statistical power due to larger standard errors as mentioned above, the possibility of meaningful bias 
cannot be ruled out by the lack of statistically significant results. 
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4. RESULTS—TIMSS ADVANCED PHYSICS 


4.1 Original Respondent Sample 


This section presents the nonresponse bias analysis based on the original sample of 237 eligible schools 
for TIMSS-P. The distribution of the participating original sample was compared to the schools in the 
total eligible original sample. School base weights were used for both the eligible sample and the 
participating schools. The unweighted school response rate for TIMSS-P was 65.8 percent before 
replacement, with 156 out of 237 schools participating. The weighted response rate was 64.9 percent 
before replacement. 


4.1.1 Categorical Variables (TIMSS-P) 


The distribution of participating and eligible schools by the four characteristics is shown in table G-21. 
There are no statistically significant relationships between participation status and any of the 
characteristics shown in table G-21. However, the absolute value of the relative bias for non-AP, private, 
central city, suburb, town, rural, Northeast, Midwest, South, and high poverty is greater than 10 percent, 
which indicates potential bias for AP schools, school control, locale, Census region, and poverty level, 
respectively (table G-21). Note that the relative bias for private and non-AP schools is much higher than 
for public and AP schools, respectively, due to the binary nature of the variable as the absolute bias is the 
same for both categories. 


Table G-21. Percentage distribution of eligible and participating schools in the U.S. TIMSS advanced 
physics original sample, by selected categorical variables: 2015 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=237) (N=156) Bias bias p value 

AP status 0.316 
Non-AP school 4.9 3:5 -1.44 -0.292 
AP school 95.1 96.5 1.44 0.015 

School control 0.673 
Public 91.2 92.3 1.07 0.012 
Private 8.8 Leh -1.07 -0.122 

Locale 0.082 
Central city 32.7 37.6 4.89 0.150 
Suburb 52.1 46.2 -5.93 -0.114 
Town 4.1 2.6 -1.54 -0.373 
Rural 11.1 13.6 2.58 0.233 

Census region 0.122 
Northeast 18.8 15.5 -3.27 -0.174 
Midwest 22.9 19.8 -3.09 -0.135 
South 31.4 39.0 7.61 0.243 
West 27.0 25.7 -1.24 -0.046 


See notes at end of table. 
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Table G-21. Percentage distribution of eligible and participating schools in the U.S. TIMSS advanced 
physics original sample, by selected categorical variables: 2015—Continued 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=237) (N=156) Bias bias p value 

Poverty level 0.253 
High 17.6 20.1 2.43 0.138 
Low 82.4 79.9 -2.43 -0.030 


NOTE: Detail may not sum to totals because of rounding. AP status indicates whether or not the school has students who took a relevant AP test 
in 2013. Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low-poverty schools Eligible schools contained at least one advanced physics student. Participating schools agreed to have their students 
assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base 
weights that did not include a nonresponse adjustment factor, and by their estimated advanced physics enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


4.1.2 Continuous Variables (TIMSS-P) 


Summary means for each continuous variable for participating and eligible schools are shown in 

tables G-22, G-23 and G-24. No data on FRPL eligibility was available for private schools, and so private 
schools are not included in the FRPL analysis. Two eligible schools had missing values for free or 
reduced-price lunch, and these schools were dropped from the analysis in table G-24. 


There were no statistically significant differences between participating and eligible schools with respect 
to student enrollment, race/ethnicity percentage or free or reduced-price lunch (tables G-22, G-23 and 
G-24). However, the absolute value of the relative bias for advanced physics enrollment, Black, non- 
Hispanic, and multiracial is greater than 10 percent (tables G-22 and G-23, respectively). Though for 
multiracial this is due mostly to the eligible percentage being less than 3.0 percent, as the absolute bias is 
small. 


Table G-22. Mean enrollment of various characteristics for eligible and participating schools in the U.S. 
TIMSS advanced physics original sample: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) Relative t test 
Student enrollment (N=237) (N=156) Bias bias p value 
Total school 1,704.4 1,659.5 -44.81 -0.026 0.557 
Advanced physics 54.0 46.9 -7.09 -0.131 0.462 


NOTE: Eligible schools contained at least one advanced physics student. Participating schools agreed to have their students assessed. The relative 
bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights that did not 
include a nonresponse adjustment factor, and by their estimated advanced physics enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table G-23. Mean percentage of various characteristics for eligible and participating schools in the U.S. 
TIMSS advanced physics original sample, by race/ethnicity: 2015 


Sample schools 


Eligible Participating 

(mean) (mean) Relative t test 
Race/ethnicity (N=237) (N=156) Bias bias p value 
White, non-Hispanic 54.3 51.5 -2.78 -0.051 0.139 
Black, non-Hispanic 14.9 17.4 2.42 0.162 0.125 
Hispanic 19.3 20.6 1.32 0.068 0.295 
Asian 8.0 FES) -0.57 -0.071 0.403 
American Indian or 
Alaska Native 0.6 0.5 -0.04 -0.069 0.653 
Hawaiian/Pacific 
Islander 0.2 0.2 -0.02 -0.092 0.334 
Multiracial 2.7 2.4 -0.34 -0.125 0.080 


NOTE: Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Eligible schools contained at 
least one advanced physics student. Participating schools agreed to have their students assessed. The relative bias is calculated as the bias divided 
by the estimate from the eligible sample. Schools were weighted by their school base weights that did not include a nonresponse adjustment 
factor, and by their estimated advanced physics enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table G-24. Mean percentage of students eligible for free or reduced-price lunch, in eligible and 
participating public schools in the U.S. TIMSS advanced physics original sample: 2015 


Sample schools 


Eligible _— Participating 
(percent) (percent) t test 
Students (N=219) (N=146) Bias _ Relative bias p value 
Percentage of students eligible for 
free or reduced-price lunch 30.1 32.0 1.85 0.061 0.317 


a ES Ee a ee ee a ae ee ee a ee ee eS a ee eae eee 
NOTE: Information on percentage of students eligible for free or reduced-price lunch is missing for two of the 221 eligible public schools in the 
sample and one of the 147 public schools that participated. Eligible schools contained at least one advanced physics student. Participating schools 

agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated advanced physics enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


4.1.3 Logistic Regression Model (TIMSS-P) 


To examine the joint relationship of various characteristics to school nonresponse, the analysis used a 
logistic regression model with participation status as the binary dependent variable and frame 
characteristics as predictor variables. Since public and private schools were modeled together using the 
variables available for all schools, the percentage of students eligible for free or reduced-price lunch was 
not included in the main logistic regression analysis. 


Standard errors and tests of hypotheses for the full model parameter estimates are presented in table G-25 
(with six race/ethnicity variables) and table G-26 (with summed race/ethnicity percentage). Private 
schools are treated as low poverty for the categorical variable poverty level. Non-AP schools, central city, 
suburb and multiracial were significant predictors of school participation in table G-25. The negative 
parameter estimate indicates that relative to AP schools, Non-AP schools were somewhat 
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underrepresented among the participating schools and that the percentage of multiracial students in 
participating schools were smaller than in all eligible schools. The positive parameter estimate indicates 
that relative to schools in rural areas, schools in central cities and suburbs were somewhat overrepresented 
among the participating schools. The F test statistic to determine whether the race/ethnicity characteristics 
are simultaneously equal to 0 was 0.46 with a p value of 0.808, which indicates that no significant 
relationship with participation was detected. 


Central city, suburb, and South region were significant predictors of school participation in table G-26. 
The positive parameter estimates indicate that relative to schools in the West region, schools in the South 
region were somewhat overrepresented among the participating schools. The parameter estimates for 
central city and suburb remained positive as in table G-25. 


Because the percentage of students eligible for free or reduced-price lunch was not included in the main 
logistic regression analysis, a separate analysis with public schools only was conducted. To include free 
or reduced-price lunch eligibility in a model, public schools were modeled separately using a model with 
the summed race/ethnicity percentage and adding the percentage of students eligible for free or reduced- 
price lunch. Since poverty is derived from the percentage of students eligible for free or reduced-price 
lunch, an interaction term’ was also included in the model. Standard errors and tests of hypotheses for the 
full model parameter estimates are presented in table G-27. Central city and South region were significant 
predictors of school participation among public schools only. The parameter estimates for central city and 
South region remained positive as in as in tables G-25 and G-26. The model with the six race/ethnicity 
variables is not shown due to complex interactions that make the results difficult to interpret. 


Table G-25. Logistic regression model parameters (with six race/ethnicity variables) using the U.S. 
TIMSS advanced physics original school sample: 2015 


Parameter t test for Ho: 
Parameter estimate Standard error parameter = 0 p value 
Intercept 0.713 1.1091 0.6432 0.5220 
Non-AP school -2.059 1.0091 -2.0400 0.0449 
Private school -0.624 0.9843 -0.6335 0.5283 
Central city 2.059 0.8166 2.5214 0.0138 
Suburb 2.193 0.9230 2.3757 0.0201 
Town 1.330 0.7817 1.7018 0.0929 
Northeast -1.351 0.8910 -1.5166 0.1336 
Midwest -0.773 0.8513 -0.9077 0.3669 
South 0.637 0.7102 0.8970 0.3726 
High poverty 0.258 0.6617 0.3897 0.6979 
Total school enrollment 0.000 0.0003 -0.7093 0.4803 
Advanced physics enrollment -0.004 0.0041 -0.8689 0.3877 
Black, non-Hispanic 0.007 0.0139 0.5308 0.5971 
Hispanic -0.005 0.0124 -0.4148 0.6795 
Asian 0.002 0.0233 0.0959 0.9239 
American Indian or Alaska Native -0.058 0.0872 -0.6702 0.5048 
Hawaiian/Pacific Islander -0.720 0.6152 -1.1697 0.2458 
Multiracial -0.209 0.1046 -1.9980 0.0493 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated advanced physics enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


7 The interaction term can be interpreted as indicating whether the marginal effect of a one percentage-point increase in FRPL is diminished or 
amplified for schools above the 50 percent cut point, relative to those below the cut point. 
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Table G-26. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS advanced physics original school sample: 2015 


Parameter t test for HO: 
Parameter estimate Standard error parameter = 0 p value 
Intercept -0.610 1.0298 -0.5920 0.5556 
Non-AP school -1.730 0.9723 -1.7790 0.0793 
Private school -0.834 0.9485 -0.8793 0.3820 
Central city 2.005 0.8996 2.2291 0.0288 
Suburb 2.277 1.0745 2.1189 0.0374 
Town 1.201 0.8699 1.3811 0.1714 
Northeast -0.260 0.6605 -0.3938 0.6949 
Midwest -0.019 0.6946 -0.0275 0.9781 
South 1.218 0.6011 2.0261 0.0463 
High poverty 0.414 0.6671 0.6202 0.5370 
Total school enrollment 0.000 0.0003 -0.9251 0.3579 
Advanced physics enrollment -0.002 0.0054 -0.3244 0.7465 
Summed race/ethnicity percentage 0.003 0.011 0.2453 0.8069 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated advanced physics enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table G-27. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS advanced physics original public school sample: 2015 


Parameter Standard t test for HO: 
Parameter estimate error parameter = 0 p value 
Intercept -0.230 1.3061 -0.1760 0.8608 
Non-AP school -1.633 1.0160 -1.6068 0.1123 
Central city 2.306 1.0088 2.2854 0.0251 
Suburb 1.745 1.0142 1.7202 0.0895 
Town 1.219 0.9778 1.2464 0.2165 
Northeast -0.449 0.7120 -0.6307 0.5301 
Midwest -0.155 0.7405 -0.2087 0.8352 
South 1.513 0.6497 2.3283 0.0226 
High poverty -1.633 1.8192 -0.8978 0.3722 
Free or reduced-price lunch eligibility -0.010 0.0241 -0.4296 0.6687 
High poverty * free or reduced-price lunch eligibility 0.038 0.0330 1.1644 0.2480 
Total school enrollment 0.000 0.0004 -0.7303 0.4675 
Advanced physics enrollment -0.003 0.0052 -0.5426 0.5890 
Summed race/ethnicity percentage -0.003 0.0149 -0.1854 0.8534 


NOTE: Information on percentage of students eligible for free or reduced-price lunch is missing for two of the 221 eligible public schools in the 
sample, so these two schools were dropped from the regression. Census region is the state-based region of the country (see technical notes for 
state listing). For public schools, a high poverty school is defined as one in which 50 percent or more of the students are eligible for participation 
in the FRPL program. Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated advanced physics enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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4.2 Participating Final Sample with Substitutes 


This section presents the nonresponse bias analysis based on the final sample of 237 eligible schools for 
TIMSS-P including participating substitute schools. The distribution of the participating final sample was 
compared to the schools in the total eligible final sample. The total eligible final sample includes 
participating final sample plus those original nonrespondents who were not replaced by substitutes. 
School base weights were used for both the eligible sample and the participating schools. Through the use 
of substitute schools, the unweighted school response rate for TIMSS-P was 69.6 percent after 
replacement, with 165 out of 237 schools participating. The weighted response rate was also 67.6 percent 
after replacement. 


4.2.1 Categorical Variables (TIMSS-P) 


The distribution of participating and eligible schools by the four characteristics is shown in table G-28. 
There are no statistically significant relationships between participation status and any of the 
characteristics shown in table G-28. However, the absolute value of the relative bias for non-AP, private, 
central city, suburb, town, rural, Northeast, Midwest, South, and high poverty is greater than 10 percent, 
which indicates potential bias for AP schools, school control, locale, Census region, and poverty level, 
respectively (table G-28). Note that the relative bias for private and non-AP schools is much higher than 
for public and AP schools, respectively, due to the binary nature of the variable as the absolute bias is the 
same for both categories. 


Table G-28. Percentage distribution of eligible and participating schools in the U.S. TIMSS advanced 
physics final sample, by selected categorical variables: 2015 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=237) (N=165) Bias bias p value 

AP status 0.271 
Non-AP school 4.8 3.3 -1.52 -0.314 
AP school 95.2 96.7 1.52 0.016 

School control 0.592 
Public 91.4 92.7 1.28 0.014 
Private 8.6 7.3 -1.28 -0.149 

Locale 0.088 
Central city 33.2 38.0 4.82 0.145 
Suburb 51.4 45.2 -6.21 -0.121 
Town 4.1 2.8 -1.26 -0.308 
Rural 11.4 14.0 2.65 0.233 

Census region 0.089 
Northeast 18.5 14.8 -3.68 -0.199 
Midwest 23.1 20.4 -2.68 -0.116 
South 32.0 39.7 7.75 0.242 
West 26.5 25.1 -1.39 -0.052 


See notes at end of table. 
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Table G-28. Percentage distribution of eligible and participating schools in the U.S. TIMSS advanced 
physics final sample, by selected categorical variables: 20 15—Continued 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 

School characteristic (N=237) (N=165) Bias bias p value 

Poverty level 0.145 
High 17.0 19.8 2.85 0.168 
Low 83.0 80.2 -2.85 -0.034 


NOTE: Detail may not sum to totals because of rounding. AP status indicates whether or not the school has students who took a relevant AP test 
in 2013. Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Eligible schools contained at least one advanced physics student. Participating schools agreed to have their students 
assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base 
weights that did not include a nonresponse adjustment factor, and by their estimated advanced physics enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


4.2.2 Continuous Variables (TIMSS-P) 


Summary means for each continuous variable for participating and eligible schools are shown in 

tables G-29, G-30, and G-31. One participating school had missing values for race/ethnicity, and this 
school was dropped from the analysis in table G-30. No data on FRPL eligibility was available for private 
schools, and so private schools are not included in the FRPL analysis. Two eligible schools had missing 
values for free or reduced-price lunch, and these schools were dropped from the analysis in table G-31. 


There were no statistically significant differences between participating and eligible schools with respect 
to student enrollment, race/ethnicity percentage or free or reduced-price lunch (tables G-29, G-30, and 
G-31). However, the absolute value of the relative bias for advanced physics enrollment, Black, non- 
Hispanic, Hawaiian/Pacific Islander, and multiracial is greater than 10 percent (tables G-29 and G-30, 
respectively). Though for Hawaiian/Pacific Islander and multiracial this is due mostly to the eligible 
percentage being less than 3.0 percent, as the absolute bias is small. 


Table G-29. Mean enrollment of various characteristics for eligible and participating schools in the U.S. 
TIMSS advanced physics final sample: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) Relative t test 
Student enrollment (N=237) (N=165) Bias bias p value 
Total school 1,703.5 1,674.0 -29.42 -0.017 0.687 
Advanced physics 54.2 46.7 -7.50 -0.138 0.424 


NOTE: Eligible schools contained at least one advanced physics student. Participating schools agreed to have their students assessed. The relative 
bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school base weights that did not 
include a nonresponse adjustment factor, and by their estimated advanced physics enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table G-30. Mean percentage of various characteristics for eligible and participating schools in the U.S. 
TIMSS advanced physics final sample, by race/ethnicity: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) Relative t test 
Race/ethnicity (N=237) (N=165) Bias bias p value 
White, non-Hispanic 55.1 52.6 -2.52 -0.046 0.156 
Black, non-Hispanic 14.6 16.9 2.26 0.154 0.119 
Hispanic 18.7 19.9 1.26 0.067 0.278 
Asian 8.1 1S -0.61 -0.076 0.347 
American Indian or Alaska 
Native 0.6 0.5 -0.05 -0.086 0.567 
Hawaiian/Pacific Islander 0.2 0.2 -0.02 -0.111 0.234 
Multiracial 2.8 205 -0.31 -0.111 0.097 


NOTE: Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Eligible schools contained at 
least one advanced physics student. Participating schools agreed to have their students assessed. The relative bias is calculated as the bias divided 
by the estimate from the eligible sample. Schools were weighted by their school base weights that did not include a nonresponse adjustment 
factor, and by their estimated advanced physics enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table G-31. Mean percentage of students eligible for free or reduced-price lunch, in eligible and 
participating public schools in the U.S. TIMSS advanced physics final sample: 2015 


Sample schools 

Eligible _— Participating 
(percent) (percent) Relative t test 

Students (N=219) (N=155) Bias bias p value 
Percentage of students eligible for 

free or reduced-price lunch 29.5 31.3 1.84 0.063 0.284 

Ga a a i ee 

NOTE: Information on percentage of students eligible for free or reduced-price lunch is missing for two of the 221 eligible public schools in the 
sample and one of the 156 public schools that participated. Eligible schools contained at least one advanced physics student. Participating schools 
agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated advanced physics enrollment. 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


4.2.3 Logistic Regression Model (TIMSS-P) 


To examine the joint relationship of various characteristics to school nonresponse, the analysis used a 
logistic regression model with participation status as the binary dependent variable and frame 
characteristics as predictor variables. Since public and private schools were modeled together using the 
variables available for all schools, the percentage of students eligible for free or reduced-price lunch was 
not included in the main logistic regression analysis. 


Standard errors and tests of hypotheses for the full model parameter estimates are presented in table G-32 
(with six race/ethnicity variables) and table G-33 (with summed race/ethnicity percentage). Private 
schools are treated as low poverty for the categorical variable poverty level. Only non-AP schools was a 
significant predictor of school participation in table G-32. The negative parameter estimate indicates that 
relative to AP schools, Non-AP schools were somewhat underrepresented among the participating schools 
The F test statistic to determine whether the race/ethnicity characteristics are simultaneously equal to 0 
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was 0.73 with a p value of 0.606, which indicates no significant relationship was detected with 
participation. 


Only the South region was a significant predictor of school participation in table G-33. The positive 
parameter estimates indicate that relative to schools in the West region, schools in the South region were 
somewhat overrepresented among the participating schools. 


Because the percentage of students eligible for free or reduced-price lunch was not included in the main 
logistic regression analysis, a separate analysis with public schools only was conducted. To include free 
or reduced-price lunch eligibility in a model, public schools were modeled separately using a model with 
the summed race/ethnicity percentage and adding the percentage of students eligible for free or reduced- 
price lunch. Since poverty is derived from the percentage of students eligible for free or reduced-price 
lunch, an interaction term was also included in the model. Standard errors and tests of hypotheses for the 
full model parameter estimates are presented in table G-34. Only the South region was a significant 
predictor of school participation among public schools only. The parameter estimates for central city and 
suburb remained positive as in table G-33.The model with the six race/ethnicity variables is not shown 
due to complex interactions that make the results difficult to interpret. 


Table G-32. Logistic regression model parameters (with six race/ethnicity variables) using the U.S. 
TIMSS advanced physics final school sample: 2015 


Parameter t test for Ho: 
Parameter estimate Standard error parameter = 0 p value 
Intercept 1.061 1.2786 0.8301 0.4091 
Non-AP school -2.201 1.0597 -2.0767 0.0413 
Private school -0.783 0.9787 -0.8000 0.4262 
Central city 1.807 0.9607 1.8806 0.0639 
Suburb 2.047 1.0718 1.9096 0.0600 
Town 1.056 0.9552 1.1057 0.2724 
Northeast -1.550 0.9230 -1.6791 0.0973 
Midwest -0.867 0.9059 -0.9566 0.3419 
South 0.606 0.7443 0.8142 0.4181 
High poverty 0.592 0.7344 0.8061 0.4228 
Total school enrollment 0.000 0.0004 -0.1786 0.8587 
Advanced physics enrollment -0.005 0.0046 -1.1783 0.2424 
Black, non-Hispanic 0.006 0.0148 0.4302 0.6683 
Hispanic -0.011 0.0131 -0.8149 0.4177 
Asian 0.001 0.0231 0.0304 0.9758 
American Indian or Alaska Native -0.079 0.0837 -0.9457 0.3473 
Hawaiian/Pacific Islander -0.916 0.6238 -1.4676 0.1464 
Multiracial -0.190 0.1004 -1.8905 0.0626 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Schools were 
weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated advanced physics enrollment. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table G-33. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS advanced physics final school sample: 2015 


Parameter t test for Ho: 
Parameter estimate Standard error parameter = 0 p value 
Intercept -0.437 1.2086 -0.3614 0.7188 
Non-AP school -1.853 1.0156 -1.8250 0.0720 
Private school -0.895 0.9528 -0.9393 0.3506 
Central city 1.843 1.0561 1.7449 0.0851 
Suburb 2.220 1.2322 1.8020 0.0756 
Town 0.977 1.0433 0.9368 0.3519 
Northeast -0.304 0.657 -0.4633 0.6445 
Midwest 0.087 0.7338 0.1190 0.9056 
South 1.323 0.6582 2.0108 0.0479 
High poverty 0.618 0.7279 0.8496 0.3982 
Total school enrollment 0.000 0.0004 -0.5050 0.6150 
Advanced physics enrollment -0.003 0.0058 -0.5163 0.6072 
Summed race/ethnicity percentage 0.001 0.0117 0.0492 0.9609 


NOTE: Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Summed race/ethnicity percentage includes Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; 
Hawaiian/Pacific Islander; and two or more races. Black includes African American, and Hispanic includes Latino. Racial categories exclude 
Hispanic origin. Schools were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated 
advanced physics enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table G-34. Logistic regression model parameters (with summed race/ethnicity percentage) using the 
U.S. TIMSS advanced physics final public school sample: 2015 


Parameter Standard t test for Ho: 
Parameter estimate error parameter = 0 p value 
Intercept -0.001 1.4879 -0.0007 0.9994 
Non-AP school -1.773 1.1151 -1.5899 0.1161 
Central city 2.139 1.1695 1.8292 0.0713 
Suburb 1.624 1.1969 1.3565 0.1790 
Town 0.972 1.1708 0.8302 0.4091 
Northeast -0.502 0.6972 -0.7198 0.4739 
Midwest -0.067 0.7737 -0.0866 0.9312 
South 1.661 0.7017 2.3676 0.0205 
High poverty -1.620 1.9755 -0.8199 0.4148 
Free or reduced-price lunch eligibility -0.013 0.0247 -0.5234 0.6022 
High poverty * free or reduced-price lunch eligibility 0.043 0.0355 1.2046 0.2322 
Total school enrollment 0.000 0.0004 -0.3221 0.7483 
Advanced physics enrollment -0.004 0.0057 -0.7540 0.4532 
Summed race/ethnicity percentage -0.001 1.4879 -0.0007 0.9994 


NOTE: Information on percentage of students eligible for free or reduced-price lunch is missing for two of the 221 eligible public schools in the 
sample, so these two schools were dropped from the regression. Census region is the state-based region of the country (see technical notes for 
state listing). For public schools, a high poverty school is defined as one in which 50 percent or more of the students are eligible for participation 
in the FRPL program. Summed race/ethnicity percentage includes Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; 
Hawaiian/Pacific Islander; and two or more races. Black includes African American, and Hispanic includes Latino. Racial categories exclude 
Hispanic origin. Schools were weighted by their school base weights that did not include a nonresponse adjustment factor, and by their estimated 
advanced physics enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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4.3 Nonresponse-adjusted Final Sample with Substitutes 


This section presents the nonresponse bias analysis based on the final sample of 237 eligible schools for 
TIMSS-P. The distribution of the participating final sample, including participating substitute schools, 
was compared to the schools in the total eligible final sample, just like the previous section. However, in 
the analyses that follow, school base weights were used for the eligible sample of schools, whereas 
nonresponse-adjusted weights were used for the participating schools. 


4.3.1 Categorical Variables (TIMSS-P) 


The distribution of participating and eligible schools by the four characteristics is shown in table G-35. 
There are no statistically significant relationships between participation status and any of the 
characteristics shown in table G-35. However, the absolute value of the relative bias for non-AP, central 
city, suburb, town, rural, and high poverty schools is greater than 10 percent, which indicates potential 
bias for AP schools, locale, and poverty level, respectively (table G-35). Note that the relative bias for 
non-AP schools is much higher than for AP schools due to the binary nature of the variable as the 
absolute bias is the same for both categories. 


Table G-35. Percentage distribution of eligible and participating schools in the U.S. TIMSS advanced 
physics nonresponse-adjusted sample, by selected categorical variables: 2015 


Sample schools 


Eligible Participating 


(percent) (percent) Relative Chi-square 
School characteristic (N=237) (N=165) Bias bias p value 
AP status 0.534 
Non-AP school 4.8 3.9 -0.98 -0.203 
AP school 95.2 96.1 0.98 0.010 
School control 0.797 
Public 91.4 92.1 0.68 0.007 
Private 8.6 7.9 -0.68 -0.079 
Locale 0.102 
Central city 33.2 37.4 4.25 0.128 
Suburb 51.4 45.0 -6.33 -0.123 
Town 4.1 2.8 -1.27 -0.312 
Rural 11.4 14.7 3.35 0.295 
Census region 0.854 
Northeast 18.5 20.1 1.62 0.088 
Midwest 23.1 20.9 -2.16 -0.094 
South 32.0 32.3 0.28 0.009 
West 26.5 26.7 0.27 0.010 
Poverty level 0.199 
High 17.0 19.6 2.66 0.157 
Low 83.0 80.4 -2.66 -0.032 


NOTE: Detail may not sum to totals because of rounding. AP status indicates whether or not the school has students who took a relevant AP test 
in 2013. Census region is the state-based region of the country (see technical notes for state listing). For public schools, a high poverty school is 
defined as one in which 50 percent or more of the students are eligible for participation in the FRPL program; all private schools are treated as 
low poverty schools. Eligible schools contained at least one advanced physics student. Participating schools agreed to have their students 
assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school 
nonresponse adjusted weight and by their estimated advanced physics enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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4.3.2 Continuous Variables (TIMSS-P) 


Summary means for each continuous variable for participating and eligible schools are shown in 

tables G-36, G-37 and G-38. No data on FRPL eligibility was available for private schools, and so private 
schools are not included in the FRPL analysis. Two eligible schools had missing values for free or 
reduced-price lunch, and these schools were dropped from the analysis in table G-31. 


There were no statistically significant differences between participating and eligible schools with respect 
to student enrollment (table G-36). Participating schools had a lower mean percentage of multiracial 
students than the eligible sample (2.4 vs. 2.8 percent, respectively; table G-37). There were no statistically 
significant differences between participating and eligible schools with respect to free or reduced-price 
lunch (table G-38). However, the absolute value of the relative bias for advanced physics enrollment, 
American Indian or Alaskan Native, and Hawaiian/Pacific Islander is greater than 10 percent (tables G-36 
and G-37, respectively). Though for the two race/ethnicity categories this is due mostly to the eligible 
percentage being less than 3.0 percent, as the absolute bias is small. 


Table G-36. Mean enrollment of various characteristics for eligible and participating schools in the U.S. 
TIMSS advanced physics nonresponse-adjusted sample: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) Relative t test 
Student enrollment (N=237) (N=165) Bias bias p value 
Total school 1,703.5 1,627.4 -76.04 -0.045 0.327 
Advanced physics 54.2 45.3 -8.85 -0.163 0.350 


NOTE: Eligible schools contained at least one advanced physics student. Participating schools agreed to have their students assessed. The relative 
bias is calculated as the bias divided by the estimate from the eligible sample. Schools were weighted by their school nonresponse adjusted 
weight and by their estimated advanced physics enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 


Table G-37. Mean percentage of various characteristics for eligible and participating schools in the U.S. 
TIMSS advanced physics nonresponse-adjusted sample, by race/ethnicity: 2015 


Sample schools 


Eligible Participating 
(mean) (mean) Relative t test 
Race/ethnicity (N=237) (N=165) Bias bias p value 
White, non-Hispanic 55.1 54.3 -0.76 -0.014 0.670 
Black, non-Hispanic 14.6 15.6 1.02 0.070 0.480 
Hispanic 18.7 19.4 0.78 0.042 0.509 
Asian 8.1 75 -0.55 -0.068 0.426 
American Indian or Alaska 
Native 0.6 0.5 -0.06 -0.105 0.482 
Hawaiian/Pacific Islander 0.2 0.2 -0.03 -0.142 0.144 
Multiracial 2.8 2.4 -0.40 -0.142 0.035 


NOTE: Black includes African American, and Hispanic includes Latino. Racial categories exclude Hispanic origin. Eligible schools contained at 
least one advanced physics student. Participating schools agreed to have their students assessed. The relative bias is calculated as the bias divided 
by the estimate from the eligible sample. Schools were weighted by their school nonresponse adjusted weight and by their estimated advanced 
physics enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015. 
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Table G-38. Mean percentage of students eligible for free or reduced-price lunch, in eligible and 
participating public schools in the U.S. TIMSS advanced physics nonresponse-adjusted 
sample: 2015 


Sample schools 
Eligible —_— Participating 
(percent) (percent) Relative t test 
Students (N=219) (N=155) Bias bias p value 
Percentage of students eligible for 
free or reduced-price lunch 29.5 30.9 1.43 0.049 0.420 


a a an a Se a a ee a a ee ee 
NOTE: Information on percentage of students eligible for free or reduced-price lunch is missing for two of the 221 participating public schools in 

the sample and one of the 156 public schools that participated. Eligible schools contained at least one advanced physics student. Participating 

schools agreed to have their students assessed. The relative bias is calculated as the bias divided by the estimate from the eligible sample. Schools 

were weighted by their school nonresponse adjusted weights and by their estimated advanced physics enrollment. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 

Study (TIMSS) 2015. 


4.4 Summary—Advanced Physics 


The investigation into nonresponse bias at the school level for the U.S. TIMSS-P effort shows statistically 
significant relationship between response status and some of the available school characteristics that were 
examined in the analysis. 


For original sample schools, no variables were found to be statistically significantly related to 
participation in the bivariate analysis. However, the absolute value of the relative bias for non-AP, 
private, central city, suburb, town, rural, Northeast, Midwest, South, high poverty, advanced physics 
enrollment, Black, non-Hispanic, and multiracial is greater than 10 percent (tables G-21, G-22, and G-23, 
respectively), which indicates potential bias even though no statistically significant relationship was 
detected. Although each of these findings indicates some potential for nonresponse bias, when all of these 
factors were considered simultaneously in a regression analysis, Non-AP schools, central city, suburb, 
and multiracial were significant predictors of participation (table G-25). The second model showed that 
Central city, suburb, and South region were significant predictors of participation (table G-26, with 
summed race/ethnicity percentage). The third model showed Central city and South region were 
significant predictors of school participation among public schools only (table G-27). 


For final sample schools (with substitutes), no variables were found to be statistically significantly related 
to participation in the bivariate analysis. However, the absolute value of the relative bias for non-AP, 
private, central city, suburb, town, rural, Northeast, Midwest, South, high poverty, advanced physics 
enrollment, Black, non-Hispanic, Hawaiian/Pacific Islander, and multiracial is greater than 10 percent 
(tables G-28, G-29, and G-30, respectively), which indicates potential bias even though no statistically 
significant relationship was detected. Though for Hawaiian/Pacific Islander and multiracial this is due 
mostly to the eligible percentage being less than 3.0 percent, as the absolute bias is small. When all of 
these factors were considered simultaneously in a regression analysis, only non-AP was a significant 
predictor of participation (table G-32). The second model showed that only South region was a significant 
predictor of participation (table G-33, with summed race/ethnicity percentage). The third model showed 
again only South region was a significant predictor of school participation among public schools only 
(table G-34). 


For final sample schools with school nonresponse adjustments applied to the weights, multiracial students 
(table G-37) were found to be statistically significantly related to participation in the bivariate analysis. 
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However, the absolute value of the relative bias for non-AP, central city, suburb, town, rural, high 
poverty, advanced physics enrollment, American Indian or Alaskan Native, and Hawaiian/Pacific Islander 
is greater than 10 percent, which indicates potential bias even though no statistically significant 
relationship was detected (tables G-35, G-36, and G-37, respectively). Though for the three race/ethnicity 
categories this is due mostly to the eligible percentage being less than 3.0 percent, as the absolute bias is 
small. The multivariate regression analysis cannot be conducted after the school nonresponse adjustments 
are applied to the weights. The concept of nonresponse-adjusted weights does not apply to the 
nonresponding units, and, thus, we cannot conduct an analysis that compares respondents with 
nonrespondents using nonresponse-adjusted weights. 


The results of the analyses are summarized in table G-39. 


Table G-39. 


Characteristics with p values less than 0.05 and absolute relative bias greater than 10 


percent, U.S. TIMSS advanced physics schools: 2015 


Analysis 
Original sample 


Regression model a 
Regression model b 
Regression model c 


Sample with substitutes 


Regression model a 
Regression model b 
Regression model c 


Nonresponse adjusted 


+ Not applicable. 


Characteristics with p values 
less than 0.05 


None 


Non-AP, central city, suburb, multiracial 
Central city, suburb, South region 
Central city, South region 


None 


Non-AP 
South region 
South region 


Multiracial 


Additional characteristics with absolute 
relative bias greater than 10 percent 
Non-AP, private, central city, suburb, 
town, rural, Northeast, Midwest, South, 
high poverty, advanced physics 
enrollment, Black, non-Hispanic, 
multiracial 


t 
+ 
t 


Non-AP, private, central city, suburb, 
town, rural, Northeast, Midwest, South, 
high poverty, advanced physics 
enrollment, Black, non-Hispanic, 
Hawaiian/Pacific Islander, multiracial 


+ 
+ 
+ 


Non-AP, central city, suburb, town, rural, 
high poverty, advanced physics 
enrollment, American Indian or Alaskan 
Native, Hawaiian/Pacific Islander 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS) 2015. 


Note that the standard errors for both advanced math and physics are generally much larger than for 
grades 4 and 8. This means some large biases could have been significant had the standard errors been in 
line with grades 4 and 8. This was likely due to having a measure of size for school sampling (grade 12 
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enrollment) that was not a good measure of the true number of advanced students. Additionally, the 
estimated advanced math and physics enrollments (used as the measure of school size for conducting 
these nonresponse bias analyses) were not always a good measure of the true number of advanced 
students. 


These results suggest that there is some potential for nonresponse bias in the U.S. TIMSS-P original 
sample based on the characteristics studied. It also suggests that, while there is some evidence that the use 
of substitute schools reduced the potential for bias, it has not reduced it substantially. Moreover, after the 
application of school nonresponse adjustments, there is some evidence of resulting potential bias in the 
available frame variables and correlated variables in the final sample with the largest bias in locale. It is 
important to note that the relative bias for private and non-AP schools is much higher than for public and 
AP schools, respectively, due to the binary nature of the variable as the absolute bias is the same for both 
categories. Also, the relative bias is potentially misleading for the American Indian or Alaskan Native, 
and Hawaiian/Pacific Islander, and multiracial categories, as the absolute bias is small in each case. Given 
there is limited statistical power due to larger standard errors as mentioned above, the possibility of 
meaningful bias cannot be ruled out by the lack of statistically significant results. 
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5. TECHNICAL NOTES 


Description of Variables 


Frame characteristics for public schools were taken from the 2012-13 CCD and, for private schools, from 


the 2011-12 PSS. 


Race/Ethnicity 


Students’ race/ethnicity was obtained through student responses to a two-part question. Students were 
asked first whether they were Hispanic or Latino, and then asked whether they were members of the 
following racial groups: American Indian/Alaska Native; Asian; Black, non-Hispanic; Native Hawaiian or 
other Pacific Islander; or White, non-Hispanic. Two or more races was derived when a student chooses 
more than one of the racial groups. The summed race/ethnicity percentage was derived from summing the 
six race/ethnicities of Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; 
Hawaiian/Pacific Islander and two or more races. 


Locale 


Locale was derived from the urban-centric locale code that is based on the urbanicity of the school 


location. 


Central city consists of a large territory inside an urbanized area and inside a principal 
city with population of 250,000 or more, midsize territory inside an urbanized area 
and inside a principal city with a population less than 250,000 and greater than or 
equal to 100,000, or small territory inside an urbanized area and inside a principal city 
with a population less than 100,000. 


Suburb consists of a large territory outside a principal city and inside an urbanized 
area with population of 250,000 or more, midsize territory outside a principal city and 
inside an urbanized area with a population less than 250,000 and greater than or equal 
to 100,000, or small territory outside a principal city and inside an urbanized area with 
a population less than 100,000. 


Town consists of a fringe territory inside an urban cluster that is less than or equal to 
10 miles from an urbanized area, distant territory inside an urban cluster that is more 
than 10 miles and less than or equal to 35 miles from an urbanized area, or remote 
territory inside an urban cluster that is more than 35 miles from an urbanized area. 


Rural consists of a fringe census-defined rural territory that is less than or equal to 5 
miles from an urbanized area, as well as rural territory that is less than or equal to 2.5 
miles from an urban cluster, distant census-defined rural territory that is more than 5 
miles but less than or equal to 25 miles from an urbanized area, as well as rural 
territory that is more than 2.5 miles but less than or equal to 10 miles from an urban 
cluster, or remote census-defined rural territory that is more than 25 miles from an 
urbanized area and is also more than 10 miles from an urban cluster. 
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Census Region 


Region is the census region of the United States. Northeast consists of Connecticut, Maine, 
Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, and Vermont. 
Midwest consists of Illinois, Indiana, lowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North 
Dakota, Ohio, South Dakota, and Wisconsin. South consists of Alabama, Arkansas, Delaware, District of 
Columbia, Florida, Georgia, Kentucky, Louisiana, Maryland, Mississippi, North Carolina, Oklahoma, 
South Carolina, Tennessee, Texas, Virginia, and West Virginia. West consists of Alaska, Arizona, 
California, Colorado, Hawaii, Idaho, Montana, Nevada, New Mexico, Oregon, Utah, Washington, and 
Wyoming. 


Percentage of Students Eligible for Free or Reduced-price Lunch 


The proportion of students in a school eligible for the free or reduced-price lunch (FRPL) program, a 
federally assisted meal program under the National School Lunch Act that provides nutritionally 
balanced, low-cost or free lunches to eligible children each school day. The question on the CCD 
questionnaire asked what percentage of students at the school were eligible to receive free or reduced- 
price lunch through the FRPL program around October 1, 2012. It is available only for public schools as 
the NCES Private School Universe Survey (PSS) data do not provide the same information for private 
schools. 


Poverty Level in Public Schools 


The measure of school poverty is based on the percentage of students eligible for FRPL. Schools were 
classified as low poverty if less than 50 percent of the students were eligible for FRPL and as high poverty 
if 50 percent or more of the students were eligible. In the interest of retaining all of the schools and 
students in these analyses, private schools were assumed to be low-poverty schools—that is, they were 
assumed to be schools in which less than 50 percent of students were eligible for FRPL. 


AP Status 


Additionally, NCES worked with the College Board to obtain data on schools that offered AP courses in 
2013. This list was matched to the school frame to supplement the frame data with school information on 
students that took AP exams. The data on the AP file included frequencies of students taking AP exams in 
calculus, physics, and both calculus and physics. Since actual counts of advanced calculus and physics 
students are not available, estimated eligible student counts were computed with the available 
information. In non-AP schools, the percentages of graduates who earned credit in calculus and/or physics 
from the 2009 High School Transcript Study (HSTS) were used with the estimated grade 12 enrollment. 
In AP schools, the counts of students taking AP exams in calculus, physics, and both calculus and physics 
were inflated based on the HSTS percentages. This was done within each school by inflating the AP 
counts in calculus, physics, and both calculus and physics by the comparable ratio of total percentage of 
advanced to AP students. For example, the number of advanced calculus students in a school was 
estimated by applying a ratio of 16.8/11.3 to the AP calculus count in the school. 
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